Note: This is test shard 3 of 6.
[==========] Running 7 tests from 2 test suites.
[----------] Global test environment set-up.
[----------] 6 tests from RaftConsensusITest
[ RUN      ] RaftConsensusITest.TestFailedOp
WARNING: Logging before InitGoogleLogging() is written to STDERR
I20260812 06:36:08.569945  4804 ts_itest-base.cc:115] Starting cluster with:
I20260812 06:36:08.570101  4804 ts_itest-base.cc:116] --------------
I20260812 06:36:08.570114  4804 ts_itest-base.cc:117] 3 tablet servers
I20260812 06:36:08.570127  4804 ts_itest-base.cc:118] 3 replicas per TS
I20260812 06:36:08.570138  4804 ts_itest-base.cc:119] --------------
2026-08-12T06:36:08Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-08-12T06:36:08Z Disabled control of system clock
I20260812 06:36:08.581168  4804 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskCnZu03/build/debug/bin/kudu
/tmp/dist-test-taskCnZu03/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestFailedOp.1786516568546459-4804-0/raft_consensus-itest-cluster/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestFailedOp.1786516568546459-4804-0/raft_consensus-itest-cluster/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestFailedOp.1786516568546459-4804-0/raft_consensus-itest-cluster/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestFailedOp.1786516568546459-4804-0/raft_consensus-itest-cluster/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.177.62:35523
--webserver_interface=127.4.177.62
--webserver_port=0
--builtin_ntp_servers=127.4.177.20:45039
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.177.62:35523 with env {}
W20260812 06:36:08.695307  4820 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:36:08.695545  4820 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:36:08.695585  4820 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 06:36:08.699440  4820 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 06:36:08.699501  4820 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:36:08.699522  4820 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 06:36:08.699538  4820 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260812 06:36:08.703380  4820 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestFailedOp.1786516568546459-4804-0/raft_consensus-itest-cluster/master-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.177.20:45039
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestFailedOp.1786516568546459-4804-0/raft_consensus-itest-cluster/master-0/data
--fs_wal_dir=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestFailedOp.1786516568546459-4804-0/raft_consensus-itest-cluster/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.177.62:35523
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.177.62:35523
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestFailedOp.1786516568546459-4804-0/raft_consensus-itest-cluster/master-0/data/info.pb
--webserver_interface=127.4.177.62
--webserver_port=0
--never_fsync=true
--heap_profile_path=/tmp/kudu.4820
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false

Master server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 06:36:08.704512  4820 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:36:08.705672  4820 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 06:36:08.711918  4825 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:36:08.712004  4827 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:36:08.712004  4820 server_base.cc:1061] running on GCE node
W20260812 06:36:08.712253  4832 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:36:08.712769  4820 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:36:08.713752  4820 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:36:08.714910  4820 hybrid_clock.cc:648] HybridClock initialized: now 1786516568714887 us; error 36 us; skew 500 ppm
I20260812 06:36:08.717088  4820 webserver.cc:533] Webserver started at http://127.4.177.62:37877/ using document root <none> and password file <none>
I20260812 06:36:08.717731  4820 fs_manager.cc:362] Metadata directory not provided
I20260812 06:36:08.717793  4820 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:36:08.717964  4820 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:36:08.719596  4820 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestFailedOp.1786516568546459-4804-0/raft_consensus-itest-cluster/master-0/data/instance:
uuid: "62c7f3d38b474b16a36c321a3c10b599"
format_stamp: "Formatted at 2026-08-12 06:36:08 on dist-test-slave-jztv"
I20260812 06:36:08.720041  4820 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestFailedOp.1786516568546459-4804-0/raft_consensus-itest-cluster/master-0/wal/instance:
uuid: "62c7f3d38b474b16a36c321a3c10b599"
format_stamp: "Formatted at 2026-08-12 06:36:08 on dist-test-slave-jztv"
I20260812 06:36:08.723439  4820 fs_manager.cc:696] Time spent creating directory manager: real 0.003s	user 0.003s	sys 0.002s
I20260812 06:36:08.725821  4844 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 06:36:08.727080  4820 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.001s
I20260812 06:36:08.727224  4820 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestFailedOp.1786516568546459-4804-0/raft_consensus-itest-cluster/master-0/data,/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestFailedOp.1786516568546459-4804-0/raft_consensus-itest-cluster/master-0/wal
uuid: "62c7f3d38b474b16a36c321a3c10b599"
format_stamp: "Formatted at 2026-08-12 06:36:08 on dist-test-slave-jztv"
I20260812 06:36:08.727332  4820 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestFailedOp.1786516568546459-4804-0/raft_consensus-itest-cluster/master-0/wal
metadata directory: /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestFailedOp.1786516568546459-4804-0/raft_consensus-itest-cluster/master-0/wal
1 data directories: /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestFailedOp.1786516568546459-4804-0/raft_consensus-itest-cluster/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:36:08.750450  4820 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 06:36:08.751269  4820 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:36:08.751502  4820 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:36:08.760416  4820 rpc_server.cc:307] RPC server started. Bound to: 127.4.177.62:35523
I20260812 06:36:08.760429  4933 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.4.177.62:35523 every 8 connection(s)
I20260812 06:36:08.761500  4820 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestFailedOp.1786516568546459-4804-0/raft_consensus-itest-cluster/master-0/data/info.pb
I20260812 06:36:08.764468  4934 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:36:08.769016  4804 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskCnZu03/build/debug/bin/kudu as pid 4820
I20260812 06:36:08.769170  4804 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestFailedOp.1786516568546459-4804-0/raft_consensus-itest-cluster/master-0/wal/instance
I20260812 06:36:08.770305  4934 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 62c7f3d38b474b16a36c321a3c10b599: Bootstrap starting.
I20260812 06:36:08.773171  4934 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 62c7f3d38b474b16a36c321a3c10b599: Neither blocks nor log segments found. Creating new log.
I20260812 06:36:08.774068  4934 log.cc:826] T 00000000000000000000000000000000 P 62c7f3d38b474b16a36c321a3c10b599: Log is configured to *not* fsync() on all Append() calls
I20260812 06:36:08.775936  4934 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 62c7f3d38b474b16a36c321a3c10b599: No bootstrap required, opened a new log
I20260812 06:36:08.779127  4934 raft_consensus.cc:359] T 00000000000000000000000000000000 P 62c7f3d38b474b16a36c321a3c10b599 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "62c7f3d38b474b16a36c321a3c10b599" member_type: VOTER last_known_addr { host: "127.4.177.62" port: 35523 } }
I20260812 06:36:08.779377  4934 raft_consensus.cc:385] T 00000000000000000000000000000000 P 62c7f3d38b474b16a36c321a3c10b599 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:36:08.779492  4934 raft_consensus.cc:740] T 00000000000000000000000000000000 P 62c7f3d38b474b16a36c321a3c10b599 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 62c7f3d38b474b16a36c321a3c10b599, State: Initialized, Role: FOLLOWER
I20260812 06:36:08.779958  4934 consensus_queue.cc:260] T 00000000000000000000000000000000 P 62c7f3d38b474b16a36c321a3c10b599 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "62c7f3d38b474b16a36c321a3c10b599" member_type: VOTER last_known_addr { host: "127.4.177.62" port: 35523 } }
I20260812 06:36:08.780114  4934 raft_consensus.cc:399] T 00000000000000000000000000000000 P 62c7f3d38b474b16a36c321a3c10b599 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 06:36:08.780215  4934 raft_consensus.cc:493] T 00000000000000000000000000000000 P 62c7f3d38b474b16a36c321a3c10b599 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 06:36:08.780354  4934 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 62c7f3d38b474b16a36c321a3c10b599 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:36:08.781380  4934 raft_consensus.cc:515] T 00000000000000000000000000000000 P 62c7f3d38b474b16a36c321a3c10b599 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "62c7f3d38b474b16a36c321a3c10b599" member_type: VOTER last_known_addr { host: "127.4.177.62" port: 35523 } }
I20260812 06:36:08.781721  4934 leader_election.cc:304] T 00000000000000000000000000000000 P 62c7f3d38b474b16a36c321a3c10b599 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 62c7f3d38b474b16a36c321a3c10b599; no voters: 
I20260812 06:36:08.782039  4934 leader_election.cc:290] T 00000000000000000000000000000000 P 62c7f3d38b474b16a36c321a3c10b599 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260812 06:36:08.782181  4941 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 62c7f3d38b474b16a36c321a3c10b599 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 06:36:08.782464  4941 raft_consensus.cc:697] T 00000000000000000000000000000000 P 62c7f3d38b474b16a36c321a3c10b599 [term 1 LEADER]: Becoming Leader. State: Replica: 62c7f3d38b474b16a36c321a3c10b599, State: Running, Role: LEADER
I20260812 06:36:08.782759  4941 consensus_queue.cc:237] T 00000000000000000000000000000000 P 62c7f3d38b474b16a36c321a3c10b599 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "62c7f3d38b474b16a36c321a3c10b599" member_type: VOTER last_known_addr { host: "127.4.177.62" port: 35523 } }
I20260812 06:36:08.783288  4934 sys_catalog.cc:565] T 00000000000000000000000000000000 P 62c7f3d38b474b16a36c321a3c10b599 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 06:36:08.784381  4943 sys_catalog.cc:455] T 00000000000000000000000000000000 P 62c7f3d38b474b16a36c321a3c10b599 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 62c7f3d38b474b16a36c321a3c10b599. Latest consensus state: current_term: 1 leader_uuid: "62c7f3d38b474b16a36c321a3c10b599" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "62c7f3d38b474b16a36c321a3c10b599" member_type: VOTER last_known_addr { host: "127.4.177.62" port: 35523 } } }
I20260812 06:36:08.784540  4943 sys_catalog.cc:458] T 00000000000000000000000000000000 P 62c7f3d38b474b16a36c321a3c10b599 [sys.catalog]: This master's current role is: LEADER
I20260812 06:36:08.784943  4951 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260812 06:36:08.785696  4942 sys_catalog.cc:455] T 00000000000000000000000000000000 P 62c7f3d38b474b16a36c321a3c10b599 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "62c7f3d38b474b16a36c321a3c10b599" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "62c7f3d38b474b16a36c321a3c10b599" member_type: VOTER last_known_addr { host: "127.4.177.62" port: 35523 } } }
I20260812 06:36:08.785856  4942 sys_catalog.cc:458] T 00000000000000000000000000000000 P 62c7f3d38b474b16a36c321a3c10b599 [sys.catalog]: This master's current role is: LEADER
I20260812 06:36:08.788465  4951 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260812 06:36:08.794090  4951 catalog_manager.cc:1383] Generated new cluster ID: 40df476cb6644ae88d2e28ba2cda89ee
I20260812 06:36:08.794170  4951 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260812 06:36:08.826138  4951 catalog_manager.cc:1406] Generated new certificate authority record
I20260812 06:36:08.827426  4951 catalog_manager.cc:1540] Loading token signing keys...
I20260812 06:36:08.835580  4951 catalog_manager.cc:6092] T 00000000000000000000000000000000 P 62c7f3d38b474b16a36c321a3c10b599: Generated new TSK 0
I20260812 06:36:08.836243  4951 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260812 06:36:08.840943  4804 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskCnZu03/build/debug/bin/kudu
/tmp/dist-test-taskCnZu03/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestFailedOp.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestFailedOp.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestFailedOp.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestFailedOp.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.177.1:0
--local_ip_for_outbound_sockets=127.4.177.1
--webserver_interface=127.4.177.1
--webserver_port=0
--tserver_master_addrs=127.4.177.62:35523
--builtin_ntp_servers=127.4.177.20:45039
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_cache_size_limit_mb=10
--consensus_rpc_timeout_ms=50 with env {}
W20260812 06:36:08.948190  4965 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:36:08.948400  4965 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:36:08.948482  4965 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 06:36:08.951732  4965 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:36:08.951860  4965 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.177.1
I20260812 06:36:08.955806  4965 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestFailedOp.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.177.20:45039
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--consensus_rpc_timeout_ms=50
--log_cache_size_limit_mb=10
--fs_data_dirs=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestFailedOp.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestFailedOp.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.177.1:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestFailedOp.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--webserver_interface=127.4.177.1
--webserver_port=0
--tserver_master_addrs=127.4.177.62:35523
--never_fsync=true
--heap_profile_path=/tmp/kudu.4965
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.177.1

Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 06:36:08.956784  4965 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:36:08.957845  4965 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 06:36:08.964069  4973 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:36:08.964043  4980 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:36:08.964051  4972 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:36:08.964485  4965 server_base.cc:1061] running on GCE node
I20260812 06:36:08.964790  4965 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:36:08.965312  4965 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:36:08.966465  4965 hybrid_clock.cc:648] HybridClock initialized: now 1786516568966444 us; error 35 us; skew 500 ppm
I20260812 06:36:08.968248  4965 webserver.cc:533] Webserver started at http://127.4.177.1:35293/ using document root <none> and password file <none>
I20260812 06:36:08.968852  4965 fs_manager.cc:362] Metadata directory not provided
I20260812 06:36:08.968909  4965 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:36:08.969113  4965 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:36:08.970675  4965 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestFailedOp.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-0/data/instance:
uuid: "273d719aa42741d19a7f3964c34bcff8"
format_stamp: "Formatted at 2026-08-12 06:36:08 on dist-test-slave-jztv"
I20260812 06:36:08.971318  4965 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestFailedOp.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-0/wal/instance:
uuid: "273d719aa42741d19a7f3964c34bcff8"
format_stamp: "Formatted at 2026-08-12 06:36:08 on dist-test-slave-jztv"
I20260812 06:36:08.975188  4965 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.000s	sys 0.004s
I20260812 06:36:08.977250  4991 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 06:36:08.978207  4965 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.001s
I20260812 06:36:08.978376  4965 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestFailedOp.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-0/data,/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestFailedOp.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-0/wal
uuid: "273d719aa42741d19a7f3964c34bcff8"
format_stamp: "Formatted at 2026-08-12 06:36:08 on dist-test-slave-jztv"
I20260812 06:36:08.978495  4965 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestFailedOp.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-0/wal
metadata directory: /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestFailedOp.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-0/wal
1 data directories: /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestFailedOp.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:36:08.992957  4965 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 06:36:08.993561  4965 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:36:08.993758  4965 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:36:08.994318  4965 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260812 06:36:08.995301  4965 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:36:08.995371  4965 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260812 06:36:08.995446  4965 ts_tablet_manager.cc:616] Registered 0 tablets
I20260812 06:36:08.995498  4965 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260812 06:36:09.007954  4965 rpc_server.cc:307] RPC server started. Bound to: 127.4.177.1:34627
I20260812 06:36:09.008004  5183 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.4.177.1:34627 every 8 connection(s)
I20260812 06:36:09.009241  4965 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestFailedOp.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-0/data/info.pb
I20260812 06:36:09.017882  5185 heartbeater.cc:344] Connected to a master server at 127.4.177.62:35523
I20260812 06:36:09.017910  4804 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskCnZu03/build/debug/bin/kudu as pid 4965
I20260812 06:36:09.018049  4804 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestFailedOp.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-0/wal/instance
I20260812 06:36:09.018294  5185 heartbeater.cc:461] Registering TS with master...
I20260812 06:36:09.018918  5185 heartbeater.cc:507] Master 127.4.177.62:35523 requested a full tablet report, sending...
I20260812 06:36:09.020633  4873 ts_manager.cc:194] Registered new tserver with Master: 273d719aa42741d19a7f3964c34bcff8 (127.4.177.1:34627)
I20260812 06:36:09.020704  4804 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskCnZu03/build/debug/bin/kudu
/tmp/dist-test-taskCnZu03/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestFailedOp.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestFailedOp.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestFailedOp.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestFailedOp.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.177.2:0
--local_ip_for_outbound_sockets=127.4.177.2
--webserver_interface=127.4.177.2
--webserver_port=0
--tserver_master_addrs=127.4.177.62:35523
--builtin_ntp_servers=127.4.177.20:45039
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_cache_size_limit_mb=10
--consensus_rpc_timeout_ms=50 with env {}
I20260812 06:36:09.022514  4873 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.4.177.1:57681
W20260812 06:36:09.130868  5191 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:36:09.131109  5191 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:36:09.131187  5191 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 06:36:09.134730  5191 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:36:09.134858  5191 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.177.2
I20260812 06:36:09.138916  5191 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestFailedOp.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-1/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.177.20:45039
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--consensus_rpc_timeout_ms=50
--log_cache_size_limit_mb=10
--fs_data_dirs=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestFailedOp.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestFailedOp.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.177.2:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestFailedOp.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--webserver_interface=127.4.177.2
--webserver_port=0
--tserver_master_addrs=127.4.177.62:35523
--never_fsync=true
--heap_profile_path=/tmp/kudu.5191
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.177.2

Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 06:36:09.140079  5191 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:36:09.141239  5191 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 06:36:09.147743  5203 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:36:09.147743  5206 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:36:09.147809  5204 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:36:09.148200  5191 server_base.cc:1061] running on GCE node
I20260812 06:36:09.148645  5191 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:36:09.149245  5191 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:36:09.150421  5191 hybrid_clock.cc:648] HybridClock initialized: now 1786516569150378 us; error 38 us; skew 500 ppm
I20260812 06:36:09.152288  5191 webserver.cc:533] Webserver started at http://127.4.177.2:37983/ using document root <none> and password file <none>
I20260812 06:36:09.152887  5191 fs_manager.cc:362] Metadata directory not provided
I20260812 06:36:09.152947  5191 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:36:09.153158  5191 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:36:09.154853  5191 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestFailedOp.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-1/data/instance:
uuid: "a64c93fd6ee34345b82ad3d4d65cd359"
format_stamp: "Formatted at 2026-08-12 06:36:09 on dist-test-slave-jztv"
I20260812 06:36:09.155504  5191 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestFailedOp.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-1/wal/instance:
uuid: "a64c93fd6ee34345b82ad3d4d65cd359"
format_stamp: "Formatted at 2026-08-12 06:36:09 on dist-test-slave-jztv"
I20260812 06:36:09.158905  5191 fs_manager.cc:696] Time spent creating directory manager: real 0.003s	user 0.003s	sys 0.001s
I20260812 06:36:09.160995  5217 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 06:36:09.161998  5191 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.001s
I20260812 06:36:09.162120  5191 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestFailedOp.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-1/data,/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestFailedOp.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-1/wal
uuid: "a64c93fd6ee34345b82ad3d4d65cd359"
format_stamp: "Formatted at 2026-08-12 06:36:09 on dist-test-slave-jztv"
I20260812 06:36:09.162263  5191 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestFailedOp.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-1/wal
metadata directory: /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestFailedOp.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-1/wal
1 data directories: /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestFailedOp.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:36:09.180548  5191 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 06:36:09.181216  5191 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:36:09.181427  5191 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:36:09.182019  5191 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260812 06:36:09.183090  5191 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:36:09.183140  5191 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260812 06:36:09.183209  5191 ts_tablet_manager.cc:616] Registered 0 tablets
I20260812 06:36:09.183248  5191 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260812 06:36:09.195016  5191 rpc_server.cc:307] RPC server started. Bound to: 127.4.177.2:43873
I20260812 06:36:09.195052  5413 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.4.177.2:43873 every 8 connection(s)
I20260812 06:36:09.195978  5191 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestFailedOp.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-1/data/info.pb
I20260812 06:36:09.199308  4804 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskCnZu03/build/debug/bin/kudu as pid 5191
I20260812 06:36:09.199885  4804 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestFailedOp.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-1/wal/instance
I20260812 06:36:09.202113  4804 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskCnZu03/build/debug/bin/kudu
/tmp/dist-test-taskCnZu03/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestFailedOp.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestFailedOp.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestFailedOp.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestFailedOp.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.177.3:0
--local_ip_for_outbound_sockets=127.4.177.3
--webserver_interface=127.4.177.3
--webserver_port=0
--tserver_master_addrs=127.4.177.62:35523
--builtin_ntp_servers=127.4.177.20:45039
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_cache_size_limit_mb=10
--consensus_rpc_timeout_ms=50 with env {}
I20260812 06:36:09.204926  5420 heartbeater.cc:344] Connected to a master server at 127.4.177.62:35523
I20260812 06:36:09.205189  5420 heartbeater.cc:461] Registering TS with master...
I20260812 06:36:09.205696  5420 heartbeater.cc:507] Master 127.4.177.62:35523 requested a full tablet report, sending...
I20260812 06:36:09.206873  4873 ts_manager.cc:194] Registered new tserver with Master: a64c93fd6ee34345b82ad3d4d65cd359 (127.4.177.2:43873)
I20260812 06:36:09.207692  4873 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.4.177.2:35455
W20260812 06:36:09.309826  5427 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:36:09.310031  5427 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:36:09.310074  5427 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 06:36:09.313446  5427 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:36:09.313562  5427 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.177.3
I20260812 06:36:09.317449  5427 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestFailedOp.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-2/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.177.20:45039
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--consensus_rpc_timeout_ms=50
--log_cache_size_limit_mb=10
--fs_data_dirs=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestFailedOp.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestFailedOp.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.177.3:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestFailedOp.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--webserver_interface=127.4.177.3
--webserver_port=0
--tserver_master_addrs=127.4.177.62:35523
--never_fsync=true
--heap_profile_path=/tmp/kudu.5427
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.177.3

Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 06:36:09.318527  5427 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:36:09.319725  5427 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 06:36:09.326260  5438 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:36:09.326269  5442 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:36:09.326287  5436 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:36:09.326565  5427 server_base.cc:1061] running on GCE node
I20260812 06:36:09.326983  5427 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:36:09.327550  5427 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:36:09.328733  5427 hybrid_clock.cc:648] HybridClock initialized: now 1786516569328719 us; error 31 us; skew 500 ppm
I20260812 06:36:09.330621  5427 webserver.cc:533] Webserver started at http://127.4.177.3:43197/ using document root <none> and password file <none>
I20260812 06:36:09.331203  5427 fs_manager.cc:362] Metadata directory not provided
I20260812 06:36:09.331287  5427 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:36:09.331519  5427 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:36:09.333058  5427 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestFailedOp.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-2/data/instance:
uuid: "352cc9b69e24462599c2069f27474730"
format_stamp: "Formatted at 2026-08-12 06:36:09 on dist-test-slave-jztv"
I20260812 06:36:09.333608  5427 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestFailedOp.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-2/wal/instance:
uuid: "352cc9b69e24462599c2069f27474730"
format_stamp: "Formatted at 2026-08-12 06:36:09 on dist-test-slave-jztv"
I20260812 06:36:09.337299  5427 fs_manager.cc:696] Time spent creating directory manager: real 0.003s	user 0.006s	sys 0.000s
I20260812 06:36:09.339712  5455 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 06:36:09.340811  5427 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.001s	sys 0.000s
I20260812 06:36:09.340969  5427 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestFailedOp.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-2/data,/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestFailedOp.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-2/wal
uuid: "352cc9b69e24462599c2069f27474730"
format_stamp: "Formatted at 2026-08-12 06:36:09 on dist-test-slave-jztv"
I20260812 06:36:09.341084  5427 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestFailedOp.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-2/wal
metadata directory: /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestFailedOp.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-2/wal
1 data directories: /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestFailedOp.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:36:09.358064  5427 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 06:36:09.358762  5427 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:36:09.358975  5427 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:36:09.359573  5427 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260812 06:36:09.360642  5427 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:36:09.360718  5427 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260812 06:36:09.360787  5427 ts_tablet_manager.cc:616] Registered 0 tablets
I20260812 06:36:09.360829  5427 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260812 06:36:09.372694  5427 rpc_server.cc:307] RPC server started. Bound to: 127.4.177.3:36375
I20260812 06:36:09.372725  5633 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.4.177.3:36375 every 8 connection(s)
I20260812 06:36:09.373659  5427 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestFailedOp.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-2/data/info.pb
I20260812 06:36:09.379746  4804 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskCnZu03/build/debug/bin/kudu as pid 5427
I20260812 06:36:09.379860  4804 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestFailedOp.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-2/wal/instance
I20260812 06:36:09.382773  5636 heartbeater.cc:344] Connected to a master server at 127.4.177.62:35523
I20260812 06:36:09.383090  5636 heartbeater.cc:461] Registering TS with master...
I20260812 06:36:09.383669  5636 heartbeater.cc:507] Master 127.4.177.62:35523 requested a full tablet report, sending...
I20260812 06:36:09.384805  4873 ts_manager.cc:194] Registered new tserver with Master: 352cc9b69e24462599c2069f27474730 (127.4.177.3:36375)
I20260812 06:36:09.385509  4873 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.4.177.3:44721
I20260812 06:36:09.394770  4804 external_mini_cluster.cc:955] 3 TS(s) registered with all masters
I20260812 06:36:09.406661  4873 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:55746:
name: "TestTable"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
owner: "alice"
W20260812 06:36:09.409040  4873 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table TestTable in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 06:36:09.426064  5077 tablet_service.cc:1511] Processing CreateTablet for tablet 361c63991009478ba808d05e8d2476d1 (DEFAULT_TABLE table=TestTable [id=e4ee1b83ee8c4ee1bc6a5cf8b10c78bf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 06:36:09.426255  5304 tablet_service.cc:1511] Processing CreateTablet for tablet 361c63991009478ba808d05e8d2476d1 (DEFAULT_TABLE table=TestTable [id=e4ee1b83ee8c4ee1bc6a5cf8b10c78bf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 06:36:09.426255  5532 tablet_service.cc:1511] Processing CreateTablet for tablet 361c63991009478ba808d05e8d2476d1 (DEFAULT_TABLE table=TestTable [id=e4ee1b83ee8c4ee1bc6a5cf8b10c78bf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 06:36:09.427121  5077 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 361c63991009478ba808d05e8d2476d1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:36:09.427150  5532 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 361c63991009478ba808d05e8d2476d1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:36:09.427150  5304 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 361c63991009478ba808d05e8d2476d1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:36:09.432328  5658 tablet_bootstrap.cc:492] T 361c63991009478ba808d05e8d2476d1 P 273d719aa42741d19a7f3964c34bcff8: Bootstrap starting.
I20260812 06:36:09.433542  5659 tablet_bootstrap.cc:492] T 361c63991009478ba808d05e8d2476d1 P a64c93fd6ee34345b82ad3d4d65cd359: Bootstrap starting.
I20260812 06:36:09.433634  5660 tablet_bootstrap.cc:492] T 361c63991009478ba808d05e8d2476d1 P 352cc9b69e24462599c2069f27474730: Bootstrap starting.
I20260812 06:36:09.434731  5658 tablet_bootstrap.cc:654] T 361c63991009478ba808d05e8d2476d1 P 273d719aa42741d19a7f3964c34bcff8: Neither blocks nor log segments found. Creating new log.
I20260812 06:36:09.435470  5659 tablet_bootstrap.cc:654] T 361c63991009478ba808d05e8d2476d1 P a64c93fd6ee34345b82ad3d4d65cd359: Neither blocks nor log segments found. Creating new log.
I20260812 06:36:09.435575  5658 log.cc:826] T 361c63991009478ba808d05e8d2476d1 P 273d719aa42741d19a7f3964c34bcff8: Log is configured to *not* fsync() on all Append() calls
I20260812 06:36:09.435628  5660 tablet_bootstrap.cc:654] T 361c63991009478ba808d05e8d2476d1 P 352cc9b69e24462599c2069f27474730: Neither blocks nor log segments found. Creating new log.
I20260812 06:36:09.436295  5659 log.cc:826] T 361c63991009478ba808d05e8d2476d1 P a64c93fd6ee34345b82ad3d4d65cd359: Log is configured to *not* fsync() on all Append() calls
I20260812 06:36:09.436357  5660 log.cc:826] T 361c63991009478ba808d05e8d2476d1 P 352cc9b69e24462599c2069f27474730: Log is configured to *not* fsync() on all Append() calls
I20260812 06:36:09.437440  5658 tablet_bootstrap.cc:492] T 361c63991009478ba808d05e8d2476d1 P 273d719aa42741d19a7f3964c34bcff8: No bootstrap required, opened a new log
I20260812 06:36:09.437633  5658 ts_tablet_manager.cc:1403] T 361c63991009478ba808d05e8d2476d1 P 273d719aa42741d19a7f3964c34bcff8: Time spent bootstrapping tablet: real 0.006s	user 0.004s	sys 0.000s
I20260812 06:36:09.437865  5660 tablet_bootstrap.cc:492] T 361c63991009478ba808d05e8d2476d1 P 352cc9b69e24462599c2069f27474730: No bootstrap required, opened a new log
I20260812 06:36:09.437942  5659 tablet_bootstrap.cc:492] T 361c63991009478ba808d05e8d2476d1 P a64c93fd6ee34345b82ad3d4d65cd359: No bootstrap required, opened a new log
I20260812 06:36:09.438062  5660 ts_tablet_manager.cc:1403] T 361c63991009478ba808d05e8d2476d1 P 352cc9b69e24462599c2069f27474730: Time spent bootstrapping tablet: real 0.005s	user 0.000s	sys 0.003s
I20260812 06:36:09.438114  5659 ts_tablet_manager.cc:1403] T 361c63991009478ba808d05e8d2476d1 P a64c93fd6ee34345b82ad3d4d65cd359: Time spent bootstrapping tablet: real 0.005s	user 0.000s	sys 0.003s
I20260812 06:36:09.440448  5660 raft_consensus.cc:359] T 361c63991009478ba808d05e8d2476d1 P 352cc9b69e24462599c2069f27474730 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "352cc9b69e24462599c2069f27474730" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 36375 } } peers { permanent_uuid: "273d719aa42741d19a7f3964c34bcff8" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 34627 } } peers { permanent_uuid: "a64c93fd6ee34345b82ad3d4d65cd359" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 43873 } }
I20260812 06:36:09.440675  5660 raft_consensus.cc:385] T 361c63991009478ba808d05e8d2476d1 P 352cc9b69e24462599c2069f27474730 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:36:09.440732  5660 raft_consensus.cc:740] T 361c63991009478ba808d05e8d2476d1 P 352cc9b69e24462599c2069f27474730 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 352cc9b69e24462599c2069f27474730, State: Initialized, Role: FOLLOWER
I20260812 06:36:09.440785  5658 raft_consensus.cc:359] T 361c63991009478ba808d05e8d2476d1 P 273d719aa42741d19a7f3964c34bcff8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "352cc9b69e24462599c2069f27474730" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 36375 } } peers { permanent_uuid: "273d719aa42741d19a7f3964c34bcff8" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 34627 } } peers { permanent_uuid: "a64c93fd6ee34345b82ad3d4d65cd359" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 43873 } }
I20260812 06:36:09.440984  5658 raft_consensus.cc:385] T 361c63991009478ba808d05e8d2476d1 P 273d719aa42741d19a7f3964c34bcff8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:36:09.441033  5658 raft_consensus.cc:740] T 361c63991009478ba808d05e8d2476d1 P 273d719aa42741d19a7f3964c34bcff8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 273d719aa42741d19a7f3964c34bcff8, State: Initialized, Role: FOLLOWER
I20260812 06:36:09.441063  5659 raft_consensus.cc:359] T 361c63991009478ba808d05e8d2476d1 P a64c93fd6ee34345b82ad3d4d65cd359 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "352cc9b69e24462599c2069f27474730" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 36375 } } peers { permanent_uuid: "273d719aa42741d19a7f3964c34bcff8" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 34627 } } peers { permanent_uuid: "a64c93fd6ee34345b82ad3d4d65cd359" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 43873 } }
I20260812 06:36:09.441143  5660 consensus_queue.cc:260] T 361c63991009478ba808d05e8d2476d1 P 352cc9b69e24462599c2069f27474730 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "352cc9b69e24462599c2069f27474730" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 36375 } } peers { permanent_uuid: "273d719aa42741d19a7f3964c34bcff8" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 34627 } } peers { permanent_uuid: "a64c93fd6ee34345b82ad3d4d65cd359" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 43873 } }
I20260812 06:36:09.441226  5659 raft_consensus.cc:385] T 361c63991009478ba808d05e8d2476d1 P a64c93fd6ee34345b82ad3d4d65cd359 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:36:09.441280  5659 raft_consensus.cc:740] T 361c63991009478ba808d05e8d2476d1 P a64c93fd6ee34345b82ad3d4d65cd359 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a64c93fd6ee34345b82ad3d4d65cd359, State: Initialized, Role: FOLLOWER
I20260812 06:36:09.441479  5658 consensus_queue.cc:260] T 361c63991009478ba808d05e8d2476d1 P 273d719aa42741d19a7f3964c34bcff8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "352cc9b69e24462599c2069f27474730" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 36375 } } peers { permanent_uuid: "273d719aa42741d19a7f3964c34bcff8" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 34627 } } peers { permanent_uuid: "a64c93fd6ee34345b82ad3d4d65cd359" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 43873 } }
I20260812 06:36:09.441673  5659 consensus_queue.cc:260] T 361c63991009478ba808d05e8d2476d1 P a64c93fd6ee34345b82ad3d4d65cd359 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "352cc9b69e24462599c2069f27474730" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 36375 } } peers { permanent_uuid: "273d719aa42741d19a7f3964c34bcff8" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 34627 } } peers { permanent_uuid: "a64c93fd6ee34345b82ad3d4d65cd359" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 43873 } }
I20260812 06:36:09.441865  5636 heartbeater.cc:499] Master 127.4.177.62:35523 was elected leader, sending a full tablet report...
I20260812 06:36:09.442032  5660 ts_tablet_manager.cc:1434] T 361c63991009478ba808d05e8d2476d1 P 352cc9b69e24462599c2069f27474730: Time spent starting tablet: real 0.004s	user 0.000s	sys 0.004s
I20260812 06:36:09.442384  5658 ts_tablet_manager.cc:1434] T 361c63991009478ba808d05e8d2476d1 P 273d719aa42741d19a7f3964c34bcff8: Time spent starting tablet: real 0.005s	user 0.005s	sys 0.000s
I20260812 06:36:09.442498  5659 ts_tablet_manager.cc:1434] T 361c63991009478ba808d05e8d2476d1 P a64c93fd6ee34345b82ad3d4d65cd359: Time spent starting tablet: real 0.004s	user 0.004s	sys 0.000s
I20260812 06:36:09.442713  5185 heartbeater.cc:499] Master 127.4.177.62:35523 was elected leader, sending a full tablet report...
I20260812 06:36:09.442776  5420 heartbeater.cc:499] Master 127.4.177.62:35523 was elected leader, sending a full tablet report...
W20260812 06:36:09.446902  5424 tablet.cc:2406] T 361c63991009478ba808d05e8d2476d1 P a64c93fd6ee34345b82ad3d4d65cd359: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 06:36:09.510455  5187 tablet.cc:2406] T 361c63991009478ba808d05e8d2476d1 P 273d719aa42741d19a7f3964c34bcff8: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 06:36:09.550382  5670 raft_consensus.cc:493] T 361c63991009478ba808d05e8d2476d1 P a64c93fd6ee34345b82ad3d4d65cd359 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 06:36:09.550635  5670 raft_consensus.cc:515] T 361c63991009478ba808d05e8d2476d1 P a64c93fd6ee34345b82ad3d4d65cd359 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "352cc9b69e24462599c2069f27474730" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 36375 } } peers { permanent_uuid: "273d719aa42741d19a7f3964c34bcff8" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 34627 } } peers { permanent_uuid: "a64c93fd6ee34345b82ad3d4d65cd359" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 43873 } }
I20260812 06:36:09.551954  5670 leader_election.cc:290] T 361c63991009478ba808d05e8d2476d1 P a64c93fd6ee34345b82ad3d4d65cd359 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 352cc9b69e24462599c2069f27474730 (127.4.177.3:36375), 273d719aa42741d19a7f3964c34bcff8 (127.4.177.1:34627)
I20260812 06:36:09.556033  5559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "361c63991009478ba808d05e8d2476d1" candidate_uuid: "a64c93fd6ee34345b82ad3d4d65cd359" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "352cc9b69e24462599c2069f27474730" is_pre_election: true
I20260812 06:36:09.556386  5559 raft_consensus.cc:2468] T 361c63991009478ba808d05e8d2476d1 P 352cc9b69e24462599c2069f27474730 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a64c93fd6ee34345b82ad3d4d65cd359 in term 0.
I20260812 06:36:09.556581  5109 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "361c63991009478ba808d05e8d2476d1" candidate_uuid: "a64c93fd6ee34345b82ad3d4d65cd359" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "273d719aa42741d19a7f3964c34bcff8" is_pre_election: true
I20260812 06:36:09.556840  5109 raft_consensus.cc:2468] T 361c63991009478ba808d05e8d2476d1 P 273d719aa42741d19a7f3964c34bcff8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a64c93fd6ee34345b82ad3d4d65cd359 in term 0.
I20260812 06:36:09.557183  5218 leader_election.cc:304] T 361c63991009478ba808d05e8d2476d1 P a64c93fd6ee34345b82ad3d4d65cd359 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 352cc9b69e24462599c2069f27474730, a64c93fd6ee34345b82ad3d4d65cd359; no voters: 
I20260812 06:36:09.557396  5670 raft_consensus.cc:2804] T 361c63991009478ba808d05e8d2476d1 P a64c93fd6ee34345b82ad3d4d65cd359 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 06:36:09.557478  5670 raft_consensus.cc:493] T 361c63991009478ba808d05e8d2476d1 P a64c93fd6ee34345b82ad3d4d65cd359 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 06:36:09.557509  5670 raft_consensus.cc:3060] T 361c63991009478ba808d05e8d2476d1 P a64c93fd6ee34345b82ad3d4d65cd359 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:36:09.558558  5670 raft_consensus.cc:515] T 361c63991009478ba808d05e8d2476d1 P a64c93fd6ee34345b82ad3d4d65cd359 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "352cc9b69e24462599c2069f27474730" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 36375 } } peers { permanent_uuid: "273d719aa42741d19a7f3964c34bcff8" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 34627 } } peers { permanent_uuid: "a64c93fd6ee34345b82ad3d4d65cd359" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 43873 } }
I20260812 06:36:09.558928  5670 leader_election.cc:290] T 361c63991009478ba808d05e8d2476d1 P a64c93fd6ee34345b82ad3d4d65cd359 [CANDIDATE]: Term 1 election: Requested vote from peers 352cc9b69e24462599c2069f27474730 (127.4.177.3:36375), 273d719aa42741d19a7f3964c34bcff8 (127.4.177.1:34627)
I20260812 06:36:09.559418  5109 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "361c63991009478ba808d05e8d2476d1" candidate_uuid: "a64c93fd6ee34345b82ad3d4d65cd359" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "273d719aa42741d19a7f3964c34bcff8"
I20260812 06:36:09.559378  5559 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "361c63991009478ba808d05e8d2476d1" candidate_uuid: "a64c93fd6ee34345b82ad3d4d65cd359" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "352cc9b69e24462599c2069f27474730"
I20260812 06:36:09.559548  5109 raft_consensus.cc:3060] T 361c63991009478ba808d05e8d2476d1 P 273d719aa42741d19a7f3964c34bcff8 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:36:09.559548  5559 raft_consensus.cc:3060] T 361c63991009478ba808d05e8d2476d1 P 352cc9b69e24462599c2069f27474730 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:36:09.560819  5559 raft_consensus.cc:2468] T 361c63991009478ba808d05e8d2476d1 P 352cc9b69e24462599c2069f27474730 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a64c93fd6ee34345b82ad3d4d65cd359 in term 1.
I20260812 06:36:09.561179  5218 leader_election.cc:304] T 361c63991009478ba808d05e8d2476d1 P a64c93fd6ee34345b82ad3d4d65cd359 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 352cc9b69e24462599c2069f27474730, a64c93fd6ee34345b82ad3d4d65cd359; no voters: 
I20260812 06:36:09.561331  5109 raft_consensus.cc:2468] T 361c63991009478ba808d05e8d2476d1 P 273d719aa42741d19a7f3964c34bcff8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a64c93fd6ee34345b82ad3d4d65cd359 in term 1.
I20260812 06:36:09.561381  5670 raft_consensus.cc:2804] T 361c63991009478ba808d05e8d2476d1 P a64c93fd6ee34345b82ad3d4d65cd359 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 06:36:09.561811  5670 raft_consensus.cc:697] T 361c63991009478ba808d05e8d2476d1 P a64c93fd6ee34345b82ad3d4d65cd359 [term 1 LEADER]: Becoming Leader. State: Replica: a64c93fd6ee34345b82ad3d4d65cd359, State: Running, Role: LEADER
I20260812 06:36:09.562142  5670 consensus_queue.cc:237] T 361c63991009478ba808d05e8d2476d1 P a64c93fd6ee34345b82ad3d4d65cd359 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "352cc9b69e24462599c2069f27474730" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 36375 } } peers { permanent_uuid: "273d719aa42741d19a7f3964c34bcff8" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 34627 } } peers { permanent_uuid: "a64c93fd6ee34345b82ad3d4d65cd359" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 43873 } }
I20260812 06:36:09.565357  4868 catalog_manager.cc:5719] T 361c63991009478ba808d05e8d2476d1 P a64c93fd6ee34345b82ad3d4d65cd359 reported cstate change: term changed from 0 to 1, leader changed from <none> to a64c93fd6ee34345b82ad3d4d65cd359 (127.4.177.2). New cstate: current_term: 1 leader_uuid: "a64c93fd6ee34345b82ad3d4d65cd359" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "352cc9b69e24462599c2069f27474730" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 36375 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "273d719aa42741d19a7f3964c34bcff8" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 34627 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a64c93fd6ee34345b82ad3d4d65cd359" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 43873 } health_report { overall_health: HEALTHY } } }
I20260812 06:36:09.574440  4804 external_mini_cluster.cc:955] 3 TS(s) registered with all masters
I20260812 06:36:09.576215  4804 ts_itest-base.cc:246] Waiting for 1 tablets on tserver 273d719aa42741d19a7f3964c34bcff8 to finish bootstrapping
I20260812 06:36:09.581996  4804 ts_itest-base.cc:246] Waiting for 1 tablets on tserver a64c93fd6ee34345b82ad3d4d65cd359 to finish bootstrapping
I20260812 06:36:09.587437  4804 ts_itest-base.cc:246] Waiting for 1 tablets on tserver 352cc9b69e24462599c2069f27474730 to finish bootstrapping
I20260812 06:36:09.604842  4804 cluster_itest_util.cc:258] Not converged past 1 yet: 0.0 1.1 0.0
W20260812 06:36:09.624689  5637 tablet.cc:2406] T 361c63991009478ba808d05e8d2476d1 P 352cc9b69e24462599c2069f27474730: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 06:36:09.708133  4804 cluster_itest_util.cc:258] Not converged past 1 yet: 0.0 1.1 0.0
I20260812 06:36:09.911347  4804 cluster_itest_util.cc:258] Not converged past 1 yet: 0.0 1.1 0.0
I20260812 06:36:09.949172  5670 consensus_queue.cc:1048] T 361c63991009478ba808d05e8d2476d1 P a64c93fd6ee34345b82ad3d4d65cd359 [LEADER]: Connected to new peer: Peer: permanent_uuid: "352cc9b69e24462599c2069f27474730" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 36375 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 06:36:09.954877  5686 consensus_queue.cc:1048] T 361c63991009478ba808d05e8d2476d1 P a64c93fd6ee34345b82ad3d4d65cd359 [LEADER]: Connected to new peer: Peer: permanent_uuid: "273d719aa42741d19a7f3964c34bcff8" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 34627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260812 06:36:10.219787  5685 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:41558: Not implemented: Unknown operation type: 115
I20260812 06:36:10.290766  5077 tablet_service.cc:1467] Tablet server has 0 leaders and 0 scanners
I20260812 06:36:10.291390  5532 tablet_service.cc:1467] Tablet server has 0 leaders and 0 scanners
I20260812 06:36:10.293044  5304 tablet_service.cc:1467] Tablet server has 1 leaders and 0 scanners
Master Summary
               UUID               |      Address       | Status
----------------------------------+--------------------+---------
 62c7f3d38b474b16a36c321a3c10b599 | 127.4.177.62:35523 | HEALTHY

Unusual flags for Master:
               Flag               |                                                                                Value                                                                                 |      Tags       |         Master
----------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------+-------------------------
 ipki_ca_key_size                 | 768                                                                                                                                                                  | experimental    | all 1 server(s) checked
 ipki_server_key_size             | 768                                                                                                                                                                  | experimental    | all 1 server(s) checked
 never_fsync                      | true                                                                                                                                                                 | unsafe,advanced | all 1 server(s) checked
 openssl_security_level_override  | 0                                                                                                                                                                    | unsafe,hidden   | all 1 server(s) checked
 rpc_reuseport                    | true                                                                                                                                                                 | experimental    | all 1 server(s) checked
 rpc_server_allow_ephemeral_ports | true                                                                                                                                                                 | unsafe          | all 1 server(s) checked
 server_dump_info_format          | pb                                                                                                                                                                   | hidden          | all 1 server(s) checked
 server_dump_info_path            | /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestFailedOp.1786516568546459-4804-0/raft_consensus-itest-cluster/master-0/data/info.pb | hidden          | all 1 server(s) checked
 tsk_num_rsa_bits                 | 512                                                                                                                                                                  | experimental    | all 1 server(s) checked

Flags of checked categories for Master:
        Flag         |       Value        |         Master
---------------------+--------------------+-------------------------
 builtin_ntp_servers | 127.4.177.20:45039 | all 1 server(s) checked
 time_source         | builtin            | all 1 server(s) checked

Tablet Server Summary
               UUID               |      Address      | Status  | Location | Tablet Leaders | Active Scanners
----------------------------------+-------------------+---------+----------+----------------+-----------------
 273d719aa42741d19a7f3964c34bcff8 | 127.4.177.1:34627 | HEALTHY | <none>   |       0        |       0
 352cc9b69e24462599c2069f27474730 | 127.4.177.3:36375 | HEALTHY | <none>   |       0        |       0
 a64c93fd6ee34345b82ad3d4d65cd359 | 127.4.177.2:43873 | HEALTHY | <none>   |       1        |       0

Tablet Server Location Summary
 Location |  Count
----------+---------
 <none>   |       3

Unusual flags for Tablet Server:
               Flag               |                                                                              Value                                                                               |      Tags       |      Tablet Server
----------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------+-------------------------
 ipki_server_key_size             | 768                                                                                                                                                              | experimental    | all 3 server(s) checked
 local_ip_for_outbound_sockets    | 127.4.177.1                                                                                                                                                      | experimental    | 127.4.177.1:34627
 local_ip_for_outbound_sockets    | 127.4.177.2                                                                                                                                                      | experimental    | 127.4.177.2:43873
 local_ip_for_outbound_sockets    | 127.4.177.3                                                                                                                                                      | experimental    | 127.4.177.3:36375
 never_fsync                      | true                                                                                                                                                             | unsafe,advanced | all 3 server(s) checked
 openssl_security_level_override  | 0                                                                                                                                                                | unsafe,hidden   | all 3 server(s) checked
 rpc_server_allow_ephemeral_ports | true                                                                                                                                                             | unsafe          | all 3 server(s) checked
 server_dump_info_format          | pb                                                                                                                                                               | hidden          | all 3 server(s) checked
 server_dump_info_path            | /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestFailedOp.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-0/data/info.pb | hidden          | 127.4.177.1:34627
 server_dump_info_path            | /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestFailedOp.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-1/data/info.pb | hidden          | 127.4.177.2:43873
 server_dump_info_path            | /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestFailedOp.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-2/data/info.pb | hidden          | 127.4.177.3:36375

Flags of checked categories for Tablet Server:
        Flag         |       Value        |      Tablet Server
---------------------+--------------------+-------------------------
 builtin_ntp_servers | 127.4.177.20:45039 | all 3 server(s) checked
 time_source         | builtin            | all 3 server(s) checked

Version Summary
     Version     |         Servers
-----------------+-------------------------
 1.19.0-SNAPSHOT | all 4 server(s) checked

Tablet Summary
The cluster doesn't have any matching system tables
Summary by table
   Name    | RF | Status  | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
-----------+----+---------+---------------+---------+------------+------------------+-------------
 TestTable | 3  | HEALTHY | 1             | 1       | 0          | 0                | 0

Tablet Replica Count Summary
   Statistic    | Replica Count
----------------+---------------
 Minimum        | 1
 First Quartile | 1
 Median         | 1
 Third Quartile | 1
 Maximum        | 1

Total Count Summary
                | Total Count
----------------+-------------
 Masters        | 1
 Tablet Servers | 3
 Tables         | 1
 Tablets        | 1
 Replicas       | 3

==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set

OK
I20260812 06:36:10.319119  4804 log_verifier.cc:126] Checking tablet 361c63991009478ba808d05e8d2476d1
I20260812 06:36:10.344940  4804 log_verifier.cc:177] Verified matching terms for 2 ops in tablet 361c63991009478ba808d05e8d2476d1
I20260812 06:36:10.620784  4804 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskCnZu03/build/debug/bin/kudu with pid 4965
I20260812 06:36:10.640826  4804 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskCnZu03/build/debug/bin/kudu with pid 5191
I20260812 06:36:10.660794  4804 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskCnZu03/build/debug/bin/kudu with pid 5427
I20260812 06:36:10.679647  4804 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskCnZu03/build/debug/bin/kudu with pid 4820
2026-08-12T06:36:10Z chronyd exiting
[       OK ] RaftConsensusITest.TestFailedOp (2154 ms)
[ RUN      ] RaftConsensusITest.MultiThreadedInsertWithFailovers
I20260812 06:36:10.714654  4804 ts_itest-base.cc:115] Starting cluster with:
I20260812 06:36:10.714715  4804 ts_itest-base.cc:116] --------------
I20260812 06:36:10.714731  4804 ts_itest-base.cc:117] 7 tablet servers
I20260812 06:36:10.714743  4804 ts_itest-base.cc:118] 7 replicas per TS
I20260812 06:36:10.714753  4804 ts_itest-base.cc:119] --------------
2026-08-12T06:36:10Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-08-12T06:36:10Z Disabled control of system clock
I20260812 06:36:10.724632  4804 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskCnZu03/build/debug/bin/kudu
/tmp/dist-test-taskCnZu03/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786516568546459-4804-0/raft_consensus-itest-cluster/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786516568546459-4804-0/raft_consensus-itest-cluster/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786516568546459-4804-0/raft_consensus-itest-cluster/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786516568546459-4804-0/raft_consensus-itest-cluster/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.177.62:42183
--webserver_interface=127.4.177.62
--webserver_port=0
--builtin_ntp_servers=127.4.177.20:34335
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.177.62:42183 with env {}
W20260812 06:36:10.828567  5761 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:36:10.828769  5761 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:36:10.828805  5761 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 06:36:10.831969  5761 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 06:36:10.832021  5761 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:36:10.832041  5761 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 06:36:10.832058  5761 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260812 06:36:10.835711  5761 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786516568546459-4804-0/raft_consensus-itest-cluster/master-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.177.20:34335
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786516568546459-4804-0/raft_consensus-itest-cluster/master-0/data
--fs_wal_dir=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786516568546459-4804-0/raft_consensus-itest-cluster/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.177.62:42183
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.177.62:42183
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786516568546459-4804-0/raft_consensus-itest-cluster/master-0/data/info.pb
--webserver_interface=127.4.177.62
--webserver_port=0
--never_fsync=true
--heap_profile_path=/tmp/kudu.5761
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false

Master server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 06:36:10.836709  5761 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:36:10.837741  5761 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 06:36:10.843150  5773 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:36:10.843307  5761 server_base.cc:1061] running on GCE node
W20260812 06:36:10.843149  5770 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:36:10.843151  5768 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:36:10.843892  5761 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:36:10.844756  5761 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:36:10.845950  5761 hybrid_clock.cc:648] HybridClock initialized: now 1786516570845930 us; error 36 us; skew 500 ppm
I20260812 06:36:10.847785  5761 webserver.cc:533] Webserver started at http://127.4.177.62:40619/ using document root <none> and password file <none>
I20260812 06:36:10.848376  5761 fs_manager.cc:362] Metadata directory not provided
I20260812 06:36:10.848465  5761 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:36:10.848706  5761 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:36:10.850317  5761 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786516568546459-4804-0/raft_consensus-itest-cluster/master-0/data/instance:
uuid: "f6078ff727624d5c90a44fd1f23a510c"
format_stamp: "Formatted at 2026-08-12 06:36:10 on dist-test-slave-jztv"
I20260812 06:36:10.850816  5761 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786516568546459-4804-0/raft_consensus-itest-cluster/master-0/wal/instance:
uuid: "f6078ff727624d5c90a44fd1f23a510c"
format_stamp: "Formatted at 2026-08-12 06:36:10 on dist-test-slave-jztv"
I20260812 06:36:10.854167  5761 fs_manager.cc:696] Time spent creating directory manager: real 0.003s	user 0.002s	sys 0.001s
I20260812 06:36:10.856353  5785 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 06:36:10.857364  5761 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.000s	sys 0.002s
I20260812 06:36:10.857501  5761 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786516568546459-4804-0/raft_consensus-itest-cluster/master-0/data,/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786516568546459-4804-0/raft_consensus-itest-cluster/master-0/wal
uuid: "f6078ff727624d5c90a44fd1f23a510c"
format_stamp: "Formatted at 2026-08-12 06:36:10 on dist-test-slave-jztv"
I20260812 06:36:10.857621  5761 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786516568546459-4804-0/raft_consensus-itest-cluster/master-0/wal
metadata directory: /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786516568546459-4804-0/raft_consensus-itest-cluster/master-0/wal
1 data directories: /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786516568546459-4804-0/raft_consensus-itest-cluster/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:36:10.873342  5761 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 06:36:10.874061  5761 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:36:10.874290  5761 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:36:10.882302  5761 rpc_server.cc:307] RPC server started. Bound to: 127.4.177.62:42183
I20260812 06:36:10.882332  5871 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.4.177.62:42183 every 8 connection(s)
I20260812 06:36:10.883455  5761 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786516568546459-4804-0/raft_consensus-itest-cluster/master-0/data/info.pb
I20260812 06:36:10.886799  5872 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:36:10.890725  4804 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskCnZu03/build/debug/bin/kudu as pid 5761
I20260812 06:36:10.890892  4804 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786516568546459-4804-0/raft_consensus-itest-cluster/master-0/wal/instance
I20260812 06:36:10.892704  5872 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f6078ff727624d5c90a44fd1f23a510c: Bootstrap starting.
I20260812 06:36:10.895022  5872 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f6078ff727624d5c90a44fd1f23a510c: Neither blocks nor log segments found. Creating new log.
I20260812 06:36:10.895753  5872 log.cc:826] T 00000000000000000000000000000000 P f6078ff727624d5c90a44fd1f23a510c: Log is configured to *not* fsync() on all Append() calls
I20260812 06:36:10.897552  5872 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f6078ff727624d5c90a44fd1f23a510c: No bootstrap required, opened a new log
I20260812 06:36:10.901039  5872 raft_consensus.cc:359] T 00000000000000000000000000000000 P f6078ff727624d5c90a44fd1f23a510c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6078ff727624d5c90a44fd1f23a510c" member_type: VOTER last_known_addr { host: "127.4.177.62" port: 42183 } }
I20260812 06:36:10.901248  5872 raft_consensus.cc:385] T 00000000000000000000000000000000 P f6078ff727624d5c90a44fd1f23a510c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:36:10.901352  5872 raft_consensus.cc:740] T 00000000000000000000000000000000 P f6078ff727624d5c90a44fd1f23a510c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6078ff727624d5c90a44fd1f23a510c, State: Initialized, Role: FOLLOWER
I20260812 06:36:10.901759  5872 consensus_queue.cc:260] T 00000000000000000000000000000000 P f6078ff727624d5c90a44fd1f23a510c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6078ff727624d5c90a44fd1f23a510c" member_type: VOTER last_known_addr { host: "127.4.177.62" port: 42183 } }
I20260812 06:36:10.901903  5872 raft_consensus.cc:399] T 00000000000000000000000000000000 P f6078ff727624d5c90a44fd1f23a510c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 06:36:10.901985  5872 raft_consensus.cc:493] T 00000000000000000000000000000000 P f6078ff727624d5c90a44fd1f23a510c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 06:36:10.902110  5872 raft_consensus.cc:3060] T 00000000000000000000000000000000 P f6078ff727624d5c90a44fd1f23a510c [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:36:10.902922  5872 raft_consensus.cc:515] T 00000000000000000000000000000000 P f6078ff727624d5c90a44fd1f23a510c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6078ff727624d5c90a44fd1f23a510c" member_type: VOTER last_known_addr { host: "127.4.177.62" port: 42183 } }
I20260812 06:36:10.903261  5872 leader_election.cc:304] T 00000000000000000000000000000000 P f6078ff727624d5c90a44fd1f23a510c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f6078ff727624d5c90a44fd1f23a510c; no voters: 
I20260812 06:36:10.903565  5872 leader_election.cc:290] T 00000000000000000000000000000000 P f6078ff727624d5c90a44fd1f23a510c [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260812 06:36:10.903702  5879 raft_consensus.cc:2804] T 00000000000000000000000000000000 P f6078ff727624d5c90a44fd1f23a510c [term 1 FOLLOWER]: Leader election won for term 1
I20260812 06:36:10.903931  5879 raft_consensus.cc:697] T 00000000000000000000000000000000 P f6078ff727624d5c90a44fd1f23a510c [term 1 LEADER]: Becoming Leader. State: Replica: f6078ff727624d5c90a44fd1f23a510c, State: Running, Role: LEADER
I20260812 06:36:10.904229  5879 consensus_queue.cc:237] T 00000000000000000000000000000000 P f6078ff727624d5c90a44fd1f23a510c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6078ff727624d5c90a44fd1f23a510c" member_type: VOTER last_known_addr { host: "127.4.177.62" port: 42183 } }
I20260812 06:36:10.904809  5872 sys_catalog.cc:565] T 00000000000000000000000000000000 P f6078ff727624d5c90a44fd1f23a510c [sys.catalog]: configured and running, proceeding with master startup.
I20260812 06:36:10.905361  5883 sys_catalog.cc:455] T 00000000000000000000000000000000 P f6078ff727624d5c90a44fd1f23a510c [sys.catalog]: SysCatalogTable state changed. Reason: New leader f6078ff727624d5c90a44fd1f23a510c. Latest consensus state: current_term: 1 leader_uuid: "f6078ff727624d5c90a44fd1f23a510c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6078ff727624d5c90a44fd1f23a510c" member_type: VOTER last_known_addr { host: "127.4.177.62" port: 42183 } } }
I20260812 06:36:10.905504  5883 sys_catalog.cc:458] T 00000000000000000000000000000000 P f6078ff727624d5c90a44fd1f23a510c [sys.catalog]: This master's current role is: LEADER
I20260812 06:36:10.905810  5881 sys_catalog.cc:455] T 00000000000000000000000000000000 P f6078ff727624d5c90a44fd1f23a510c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f6078ff727624d5c90a44fd1f23a510c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6078ff727624d5c90a44fd1f23a510c" member_type: VOTER last_known_addr { host: "127.4.177.62" port: 42183 } } }
I20260812 06:36:10.905897  5881 sys_catalog.cc:458] T 00000000000000000000000000000000 P f6078ff727624d5c90a44fd1f23a510c [sys.catalog]: This master's current role is: LEADER
I20260812 06:36:10.906363  5886 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260812 06:36:10.909417  5886 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260812 06:36:10.915510  5886 catalog_manager.cc:1383] Generated new cluster ID: c5a254b3187f416b86aa5e64a204caa5
I20260812 06:36:10.915608  5886 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260812 06:36:10.932343  5886 catalog_manager.cc:1406] Generated new certificate authority record
I20260812 06:36:10.933197  5886 catalog_manager.cc:1540] Loading token signing keys...
I20260812 06:36:10.941174  5886 catalog_manager.cc:6092] T 00000000000000000000000000000000 P f6078ff727624d5c90a44fd1f23a510c: Generated new TSK 0
I20260812 06:36:10.941787  5886 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260812 06:36:10.945111  4804 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskCnZu03/build/debug/bin/kudu
/tmp/dist-test-taskCnZu03/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.177.1:0
--local_ip_for_outbound_sockets=127.4.177.1
--webserver_interface=127.4.177.1
--webserver_port=0
--tserver_master_addrs=127.4.177.62:42183
--builtin_ntp_servers=127.4.177.20:34335
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_cache_size_limit_mb=10
--consensus_rpc_timeout_ms=1000 with env {}
W20260812 06:36:11.073060  5910 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:36:11.073298  5910 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:36:11.073374  5910 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 06:36:11.076762  5910 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:36:11.076926  5910 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.177.1
I20260812 06:36:11.080760  5910 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.177.20:34335
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--consensus_rpc_timeout_ms=1000
--log_cache_size_limit_mb=10
--fs_data_dirs=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.177.1:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--webserver_interface=127.4.177.1
--webserver_port=0
--tserver_master_addrs=127.4.177.62:42183
--never_fsync=true
--heap_profile_path=/tmp/kudu.5910
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.177.1

Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 06:36:11.081904  5910 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:36:11.083127  5910 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 06:36:11.089605  5921 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:36:11.089648  5926 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:36:11.089618  5920 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:36:11.089979  5910 server_base.cc:1061] running on GCE node
I20260812 06:36:11.090512  5910 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:36:11.091099  5910 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:36:11.092288  5910 hybrid_clock.cc:648] HybridClock initialized: now 1786516571092277 us; error 30 us; skew 500 ppm
I20260812 06:36:11.094202  5910 webserver.cc:533] Webserver started at http://127.4.177.1:46031/ using document root <none> and password file <none>
I20260812 06:36:11.094856  5910 fs_manager.cc:362] Metadata directory not provided
I20260812 06:36:11.094946  5910 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:36:11.095180  5910 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:36:11.096799  5910 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-0/data/instance:
uuid: "ec6c8d818e744cdbb6b0d15b321010e0"
format_stamp: "Formatted at 2026-08-12 06:36:11 on dist-test-slave-jztv"
I20260812 06:36:11.097301  5910 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-0/wal/instance:
uuid: "ec6c8d818e744cdbb6b0d15b321010e0"
format_stamp: "Formatted at 2026-08-12 06:36:11 on dist-test-slave-jztv"
I20260812 06:36:11.100896  5910 fs_manager.cc:696] Time spent creating directory manager: real 0.003s	user 0.000s	sys 0.006s
I20260812 06:36:11.103237  5937 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 06:36:11.104300  5910 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.000s	sys 0.003s
I20260812 06:36:11.104445  5910 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-0/data,/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-0/wal
uuid: "ec6c8d818e744cdbb6b0d15b321010e0"
format_stamp: "Formatted at 2026-08-12 06:36:11 on dist-test-slave-jztv"
I20260812 06:36:11.104557  5910 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-0/wal
metadata directory: /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-0/wal
1 data directories: /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:36:11.114533  5910 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 06:36:11.115155  5910 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:36:11.115355  5910 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:36:11.115937  5910 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260812 06:36:11.116930  5910 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:36:11.117007  5910 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260812 06:36:11.117074  5910 ts_tablet_manager.cc:616] Registered 0 tablets
I20260812 06:36:11.117125  5910 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260812 06:36:11.127069  5910 rpc_server.cc:307] RPC server started. Bound to: 127.4.177.1:41057
I20260812 06:36:11.127146  6117 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.4.177.1:41057 every 8 connection(s)
I20260812 06:36:11.128089  5910 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-0/data/info.pb
I20260812 06:36:11.133364  4804 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskCnZu03/build/debug/bin/kudu as pid 5910
I20260812 06:36:11.133497  4804 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-0/wal/instance
I20260812 06:36:11.135886  4804 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskCnZu03/build/debug/bin/kudu
/tmp/dist-test-taskCnZu03/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.177.2:0
--local_ip_for_outbound_sockets=127.4.177.2
--webserver_interface=127.4.177.2
--webserver_port=0
--tserver_master_addrs=127.4.177.62:42183
--builtin_ntp_servers=127.4.177.20:34335
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_cache_size_limit_mb=10
--consensus_rpc_timeout_ms=1000 with env {}
I20260812 06:36:11.137002  6118 heartbeater.cc:344] Connected to a master server at 127.4.177.62:42183
I20260812 06:36:11.137245  6118 heartbeater.cc:461] Registering TS with master...
I20260812 06:36:11.137802  6118 heartbeater.cc:507] Master 127.4.177.62:42183 requested a full tablet report, sending...
I20260812 06:36:11.139405  5814 ts_manager.cc:194] Registered new tserver with Master: ec6c8d818e744cdbb6b0d15b321010e0 (127.4.177.1:41057)
I20260812 06:36:11.140683  5814 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.4.177.1:42097
W20260812 06:36:11.242946  6122 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:36:11.243177  6122 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:36:11.243253  6122 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 06:36:11.246692  6122 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:36:11.246821  6122 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.177.2
I20260812 06:36:11.250698  6122 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-1/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.177.20:34335
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--consensus_rpc_timeout_ms=1000
--log_cache_size_limit_mb=10
--fs_data_dirs=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.177.2:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--webserver_interface=127.4.177.2
--webserver_port=0
--tserver_master_addrs=127.4.177.62:42183
--never_fsync=true
--heap_profile_path=/tmp/kudu.6122
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.177.2

Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 06:36:11.251814  6122 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:36:11.252955  6122 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 06:36:11.259438  6134 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:36:11.259496  6136 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:36:11.259447  6133 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:36:11.259668  6122 server_base.cc:1061] running on GCE node
I20260812 06:36:11.260069  6122 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:36:11.260597  6122 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:36:11.261777  6122 hybrid_clock.cc:648] HybridClock initialized: now 1786516571261755 us; error 43 us; skew 500 ppm
I20260812 06:36:11.263538  6122 webserver.cc:533] Webserver started at http://127.4.177.2:41983/ using document root <none> and password file <none>
I20260812 06:36:11.264093  6122 fs_manager.cc:362] Metadata directory not provided
I20260812 06:36:11.264178  6122 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:36:11.264400  6122 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:36:11.265954  6122 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-1/data/instance:
uuid: "8d90d61da62549208fe117b4384a469b"
format_stamp: "Formatted at 2026-08-12 06:36:11 on dist-test-slave-jztv"
I20260812 06:36:11.266505  6122 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-1/wal/instance:
uuid: "8d90d61da62549208fe117b4384a469b"
format_stamp: "Formatted at 2026-08-12 06:36:11 on dist-test-slave-jztv"
I20260812 06:36:11.269788  6122 fs_manager.cc:696] Time spent creating directory manager: real 0.003s	user 0.002s	sys 0.001s
I20260812 06:36:11.272048  6145 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 06:36:11.273087  6122 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260812 06:36:11.273216  6122 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-1/data,/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-1/wal
uuid: "8d90d61da62549208fe117b4384a469b"
format_stamp: "Formatted at 2026-08-12 06:36:11 on dist-test-slave-jztv"
I20260812 06:36:11.273319  6122 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-1/wal
metadata directory: /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-1/wal
1 data directories: /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:36:11.316205  6122 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 06:36:11.316980  6122 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:36:11.317189  6122 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:36:11.317792  6122 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260812 06:36:11.318867  6122 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:36:11.318949  6122 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260812 06:36:11.319023  6122 ts_tablet_manager.cc:616] Registered 0 tablets
I20260812 06:36:11.319058  6122 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260812 06:36:11.329236  6122 rpc_server.cc:307] RPC server started. Bound to: 127.4.177.2:41539
I20260812 06:36:11.329290  6324 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.4.177.2:41539 every 8 connection(s)
I20260812 06:36:11.330305  6122 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-1/data/info.pb
I20260812 06:36:11.333853  4804 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskCnZu03/build/debug/bin/kudu as pid 6122
I20260812 06:36:11.333981  4804 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-1/wal/instance
I20260812 06:36:11.336352  4804 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskCnZu03/build/debug/bin/kudu
/tmp/dist-test-taskCnZu03/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.177.3:0
--local_ip_for_outbound_sockets=127.4.177.3
--webserver_interface=127.4.177.3
--webserver_port=0
--tserver_master_addrs=127.4.177.62:42183
--builtin_ntp_servers=127.4.177.20:34335
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_cache_size_limit_mb=10
--consensus_rpc_timeout_ms=1000 with env {}
I20260812 06:36:11.339711  6328 heartbeater.cc:344] Connected to a master server at 127.4.177.62:42183
I20260812 06:36:11.339985  6328 heartbeater.cc:461] Registering TS with master...
I20260812 06:36:11.340566  6328 heartbeater.cc:507] Master 127.4.177.62:42183 requested a full tablet report, sending...
I20260812 06:36:11.341768  5814 ts_manager.cc:194] Registered new tserver with Master: 8d90d61da62549208fe117b4384a469b (127.4.177.2:41539)
I20260812 06:36:11.342428  5814 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.4.177.2:59197
W20260812 06:36:11.444861  6333 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:36:11.445111  6333 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:36:11.445201  6333 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 06:36:11.448597  6333 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:36:11.448758  6333 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.177.3
I20260812 06:36:11.452661  6333 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-2/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.177.20:34335
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--consensus_rpc_timeout_ms=1000
--log_cache_size_limit_mb=10
--fs_data_dirs=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.177.3:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--webserver_interface=127.4.177.3
--webserver_port=0
--tserver_master_addrs=127.4.177.62:42183
--never_fsync=true
--heap_profile_path=/tmp/kudu.6333
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.177.3

Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 06:36:11.453797  6333 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:36:11.455004  6333 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 06:36:11.461256  6343 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:36:11.461434  6346 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:36:11.461256  6348 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:36:11.461678  6333 server_base.cc:1061] running on GCE node
I20260812 06:36:11.462075  6333 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:36:11.462623  6333 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:36:11.463773  6333 hybrid_clock.cc:648] HybridClock initialized: now 1786516571463756 us; error 28 us; skew 500 ppm
I20260812 06:36:11.465451  6333 webserver.cc:533] Webserver started at http://127.4.177.3:44527/ using document root <none> and password file <none>
I20260812 06:36:11.466003  6333 fs_manager.cc:362] Metadata directory not provided
I20260812 06:36:11.466056  6333 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:36:11.466333  6333 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:36:11.467806  6333 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-2/data/instance:
uuid: "7962f587ccd842ad8a3f6ed9a5d7707e"
format_stamp: "Formatted at 2026-08-12 06:36:11 on dist-test-slave-jztv"
I20260812 06:36:11.468292  6333 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-2/wal/instance:
uuid: "7962f587ccd842ad8a3f6ed9a5d7707e"
format_stamp: "Formatted at 2026-08-12 06:36:11 on dist-test-slave-jztv"
I20260812 06:36:11.471529  6333 fs_manager.cc:696] Time spent creating directory manager: real 0.003s	user 0.001s	sys 0.003s
I20260812 06:36:11.473598  6358 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 06:36:11.474584  6333 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260812 06:36:11.474712  6333 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-2/data,/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-2/wal
uuid: "7962f587ccd842ad8a3f6ed9a5d7707e"
format_stamp: "Formatted at 2026-08-12 06:36:11 on dist-test-slave-jztv"
I20260812 06:36:11.474812  6333 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-2/wal
metadata directory: /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-2/wal
1 data directories: /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:36:11.491801  6333 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 06:36:11.492494  6333 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:36:11.492693  6333 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:36:11.493428  6333 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260812 06:36:11.494562  6333 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:36:11.494635  6333 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260812 06:36:11.494720  6333 ts_tablet_manager.cc:616] Registered 0 tablets
I20260812 06:36:11.494772  6333 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260812 06:36:11.504963  6333 rpc_server.cc:307] RPC server started. Bound to: 127.4.177.3:40897
I20260812 06:36:11.505009  6555 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.4.177.3:40897 every 8 connection(s)
I20260812 06:36:11.505993  6333 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-2/data/info.pb
I20260812 06:36:11.514109  4804 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskCnZu03/build/debug/bin/kudu as pid 6333
I20260812 06:36:11.514246  4804 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-2/wal/instance
I20260812 06:36:11.514431  6556 heartbeater.cc:344] Connected to a master server at 127.4.177.62:42183
I20260812 06:36:11.514683  6556 heartbeater.cc:461] Registering TS with master...
I20260812 06:36:11.515288  6556 heartbeater.cc:507] Master 127.4.177.62:42183 requested a full tablet report, sending...
I20260812 06:36:11.516382  5814 ts_manager.cc:194] Registered new tserver with Master: 7962f587ccd842ad8a3f6ed9a5d7707e (127.4.177.3:40897)
I20260812 06:36:11.516382  4804 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskCnZu03/build/debug/bin/kudu
/tmp/dist-test-taskCnZu03/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-3/wal
--fs_data_dirs=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-3/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-3/logs
--server_dump_info_path=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-3/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.177.4:0
--local_ip_for_outbound_sockets=127.4.177.4
--webserver_interface=127.4.177.4
--webserver_port=0
--tserver_master_addrs=127.4.177.62:42183
--builtin_ntp_servers=127.4.177.20:34335
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_cache_size_limit_mb=10
--consensus_rpc_timeout_ms=1000 with env {}
I20260812 06:36:11.517149  5814 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.4.177.3:53107
W20260812 06:36:11.621831  6561 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:36:11.622048  6561 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:36:11.622092  6561 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 06:36:11.625281  6561 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:36:11.625382  6561 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.177.4
I20260812 06:36:11.629287  6561 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-3/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.177.20:34335
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--consensus_rpc_timeout_ms=1000
--log_cache_size_limit_mb=10
--fs_data_dirs=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-3/data
--fs_wal_dir=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-3/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.177.4:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-3/data/info.pb
--webserver_interface=127.4.177.4
--webserver_port=0
--tserver_master_addrs=127.4.177.62:42183
--never_fsync=true
--heap_profile_path=/tmp/kudu.6561
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.177.4

Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 06:36:11.630322  6561 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:36:11.631379  6561 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 06:36:11.637707  6576 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:36:11.637720  6575 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:36:11.637825  6561 server_base.cc:1061] running on GCE node
W20260812 06:36:11.637713  6578 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:36:11.638274  6561 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:36:11.638859  6561 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:36:11.640038  6561 hybrid_clock.cc:648] HybridClock initialized: now 1786516571640023 us; error 31 us; skew 500 ppm
I20260812 06:36:11.641733  6561 webserver.cc:533] Webserver started at http://127.4.177.4:34183/ using document root <none> and password file <none>
I20260812 06:36:11.642387  6561 fs_manager.cc:362] Metadata directory not provided
I20260812 06:36:11.642472  6561 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:36:11.642673  6561 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:36:11.644318  6561 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-3/data/instance:
uuid: "42895a9bcba64aa484f6509605035d02"
format_stamp: "Formatted at 2026-08-12 06:36:11 on dist-test-slave-jztv"
I20260812 06:36:11.644821  6561 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-3/wal/instance:
uuid: "42895a9bcba64aa484f6509605035d02"
format_stamp: "Formatted at 2026-08-12 06:36:11 on dist-test-slave-jztv"
I20260812 06:36:11.648260  6561 fs_manager.cc:696] Time spent creating directory manager: real 0.003s	user 0.003s	sys 0.002s
I20260812 06:36:11.650501  6593 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 06:36:11.651510  6561 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.001s
I20260812 06:36:11.651645  6561 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-3/data,/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-3/wal
uuid: "42895a9bcba64aa484f6509605035d02"
format_stamp: "Formatted at 2026-08-12 06:36:11 on dist-test-slave-jztv"
I20260812 06:36:11.651752  6561 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-3/wal
metadata directory: /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-3/wal
1 data directories: /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-3/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:36:11.668084  6561 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 06:36:11.668725  6561 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:36:11.668913  6561 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:36:11.669480  6561 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260812 06:36:11.670512  6561 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:36:11.670583  6561 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260812 06:36:11.670652  6561 ts_tablet_manager.cc:616] Registered 0 tablets
I20260812 06:36:11.670706  6561 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260812 06:36:11.680419  6561 rpc_server.cc:307] RPC server started. Bound to: 127.4.177.4:38181
I20260812 06:36:11.680506  6783 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.4.177.4:38181 every 8 connection(s)
I20260812 06:36:11.681422  6561 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-3/data/info.pb
I20260812 06:36:11.683619  4804 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskCnZu03/build/debug/bin/kudu as pid 6561
I20260812 06:36:11.683707  4804 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-3/wal/instance
I20260812 06:36:11.685827  4804 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskCnZu03/build/debug/bin/kudu
/tmp/dist-test-taskCnZu03/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-4/wal
--fs_data_dirs=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-4/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-4/logs
--server_dump_info_path=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-4/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.177.5:0
--local_ip_for_outbound_sockets=127.4.177.5
--webserver_interface=127.4.177.5
--webserver_port=0
--tserver_master_addrs=127.4.177.62:42183
--builtin_ntp_servers=127.4.177.20:34335
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_cache_size_limit_mb=10
--consensus_rpc_timeout_ms=1000 with env {}
I20260812 06:36:11.690065  6784 heartbeater.cc:344] Connected to a master server at 127.4.177.62:42183
I20260812 06:36:11.690343  6784 heartbeater.cc:461] Registering TS with master...
I20260812 06:36:11.690852  6784 heartbeater.cc:507] Master 127.4.177.62:42183 requested a full tablet report, sending...
I20260812 06:36:11.691926  5814 ts_manager.cc:194] Registered new tserver with Master: 42895a9bcba64aa484f6509605035d02 (127.4.177.4:38181)
I20260812 06:36:11.692492  5814 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.4.177.4:58427
W20260812 06:36:11.795120  6795 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:36:11.795337  6795 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:36:11.795378  6795 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 06:36:11.798626  6795 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:36:11.798730  6795 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.177.5
I20260812 06:36:11.802501  6795 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-4/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.177.20:34335
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--consensus_rpc_timeout_ms=1000
--log_cache_size_limit_mb=10
--fs_data_dirs=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-4/data
--fs_wal_dir=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-4/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.177.5:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-4/data/info.pb
--webserver_interface=127.4.177.5
--webserver_port=0
--tserver_master_addrs=127.4.177.62:42183
--never_fsync=true
--heap_profile_path=/tmp/kudu.6795
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.177.5

Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 06:36:11.803623  6795 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:36:11.804688  6795 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 06:36:11.811702  6801 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:36:11.811743  6802 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:36:11.811702  6804 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:36:11.812022  6795 server_base.cc:1061] running on GCE node
I20260812 06:36:11.812417  6795 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:36:11.812966  6795 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:36:11.814157  6795 hybrid_clock.cc:648] HybridClock initialized: now 1786516571814140 us; error 26 us; skew 500 ppm
I20260812 06:36:11.816301  6795 webserver.cc:533] Webserver started at http://127.4.177.5:43111/ using document root <none> and password file <none>
I20260812 06:36:11.816892  6795 fs_manager.cc:362] Metadata directory not provided
I20260812 06:36:11.816948  6795 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:36:11.817194  6795 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:36:11.818907  6795 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-4/data/instance:
uuid: "222b20de5a1145c6b8a11df8374e8790"
format_stamp: "Formatted at 2026-08-12 06:36:11 on dist-test-slave-jztv"
I20260812 06:36:11.819434  6795 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-4/wal/instance:
uuid: "222b20de5a1145c6b8a11df8374e8790"
format_stamp: "Formatted at 2026-08-12 06:36:11 on dist-test-slave-jztv"
I20260812 06:36:11.822903  6795 fs_manager.cc:696] Time spent creating directory manager: real 0.003s	user 0.003s	sys 0.001s
I20260812 06:36:11.825088  6816 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 06:36:11.826126  6795 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.001s
I20260812 06:36:11.826278  6795 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-4/data,/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-4/wal
uuid: "222b20de5a1145c6b8a11df8374e8790"
format_stamp: "Formatted at 2026-08-12 06:36:11 on dist-test-slave-jztv"
I20260812 06:36:11.826399  6795 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-4/wal
metadata directory: /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-4/wal
1 data directories: /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-4/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:36:11.842160  6795 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 06:36:11.842911  6795 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:36:11.843122  6795 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:36:11.843778  6795 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260812 06:36:11.844805  6795 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:36:11.844995  6795 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260812 06:36:11.845119  6795 ts_tablet_manager.cc:616] Registered 0 tablets
I20260812 06:36:11.845173  6795 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260812 06:36:11.856281  6795 rpc_server.cc:307] RPC server started. Bound to: 127.4.177.5:43521
I20260812 06:36:11.856309  6984 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.4.177.5:43521 every 8 connection(s)
I20260812 06:36:11.857335  6795 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-4/data/info.pb
I20260812 06:36:11.863845  4804 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskCnZu03/build/debug/bin/kudu as pid 6795
I20260812 06:36:11.863997  4804 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-4/wal/instance
I20260812 06:36:11.866299  6985 heartbeater.cc:344] Connected to a master server at 127.4.177.62:42183
I20260812 06:36:11.866534  4804 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskCnZu03/build/debug/bin/kudu
/tmp/dist-test-taskCnZu03/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-5/wal
--fs_data_dirs=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-5/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-5/logs
--server_dump_info_path=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-5/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.177.6:0
--local_ip_for_outbound_sockets=127.4.177.6
--webserver_interface=127.4.177.6
--webserver_port=0
--tserver_master_addrs=127.4.177.62:42183
--builtin_ntp_servers=127.4.177.20:34335
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_cache_size_limit_mb=10
--consensus_rpc_timeout_ms=1000 with env {}
I20260812 06:36:11.866564  6985 heartbeater.cc:461] Registering TS with master...
I20260812 06:36:11.867179  6985 heartbeater.cc:507] Master 127.4.177.62:42183 requested a full tablet report, sending...
I20260812 06:36:11.868391  5814 ts_manager.cc:194] Registered new tserver with Master: 222b20de5a1145c6b8a11df8374e8790 (127.4.177.5:43521)
I20260812 06:36:11.869184  5814 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.4.177.5:38523
W20260812 06:36:11.973847  6990 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:36:11.974066  6990 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:36:11.974108  6990 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 06:36:11.977423  6990 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:36:11.977519  6990 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.177.6
I20260812 06:36:11.981326  6990 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-5/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.177.20:34335
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--consensus_rpc_timeout_ms=1000
--log_cache_size_limit_mb=10
--fs_data_dirs=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-5/data
--fs_wal_dir=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-5/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.177.6:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-5/data/info.pb
--webserver_interface=127.4.177.6
--webserver_port=0
--tserver_master_addrs=127.4.177.62:42183
--never_fsync=true
--heap_profile_path=/tmp/kudu.6990
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.177.6

Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 06:36:11.982786  6990 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:36:11.984068  6990 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 06:36:11.990377  7003 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:36:11.990372  7001 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:36:11.990553  7005 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:36:11.990830  6990 server_base.cc:1061] running on GCE node
I20260812 06:36:11.991186  6990 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:36:11.991729  6990 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:36:11.992885  6990 hybrid_clock.cc:648] HybridClock initialized: now 1786516571992873 us; error 22 us; skew 500 ppm
I20260812 06:36:11.994707  6990 webserver.cc:533] Webserver started at http://127.4.177.6:42073/ using document root <none> and password file <none>
I20260812 06:36:11.995268  6990 fs_manager.cc:362] Metadata directory not provided
I20260812 06:36:11.995324  6990 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:36:11.995579  6990 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:36:11.997113  6990 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-5/data/instance:
uuid: "2df4308c852240859ada53d216ab69ae"
format_stamp: "Formatted at 2026-08-12 06:36:11 on dist-test-slave-jztv"
I20260812 06:36:11.997606  6990 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-5/wal/instance:
uuid: "2df4308c852240859ada53d216ab69ae"
format_stamp: "Formatted at 2026-08-12 06:36:11 on dist-test-slave-jztv"
I20260812 06:36:12.000945  6990 fs_manager.cc:696] Time spent creating directory manager: real 0.003s	user 0.002s	sys 0.004s
I20260812 06:36:12.003119  7014 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 06:36:12.004140  6990 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260812 06:36:12.004267  6990 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-5/data,/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-5/wal
uuid: "2df4308c852240859ada53d216ab69ae"
format_stamp: "Formatted at 2026-08-12 06:36:11 on dist-test-slave-jztv"
I20260812 06:36:12.004369  6990 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-5/wal
metadata directory: /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-5/wal
1 data directories: /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-5/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:36:12.020689  6990 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 06:36:12.021396  6990 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:36:12.021607  6990 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:36:12.022194  6990 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260812 06:36:12.023317  6990 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:36:12.023401  6990 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260812 06:36:12.023494  6990 ts_tablet_manager.cc:616] Registered 0 tablets
I20260812 06:36:12.023545  6990 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260812 06:36:12.035696  6990 rpc_server.cc:307] RPC server started. Bound to: 127.4.177.6:34269
I20260812 06:36:12.035727  7175 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.4.177.6:34269 every 8 connection(s)
I20260812 06:36:12.036725  6990 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-5/data/info.pb
I20260812 06:36:12.044358  4804 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskCnZu03/build/debug/bin/kudu as pid 6990
I20260812 06:36:12.044495  4804 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-5/wal/instance
I20260812 06:36:12.045189  7178 heartbeater.cc:344] Connected to a master server at 127.4.177.62:42183
I20260812 06:36:12.045445  7178 heartbeater.cc:461] Registering TS with master...
I20260812 06:36:12.046092  7178 heartbeater.cc:507] Master 127.4.177.62:42183 requested a full tablet report, sending...
I20260812 06:36:12.047036  4804 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskCnZu03/build/debug/bin/kudu
/tmp/dist-test-taskCnZu03/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-6/wal
--fs_data_dirs=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-6/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-6/logs
--server_dump_info_path=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-6/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.177.7:0
--local_ip_for_outbound_sockets=127.4.177.7
--webserver_interface=127.4.177.7
--webserver_port=0
--tserver_master_addrs=127.4.177.62:42183
--builtin_ntp_servers=127.4.177.20:34335
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_cache_size_limit_mb=10
--consensus_rpc_timeout_ms=1000 with env {}
I20260812 06:36:12.047322  5814 ts_manager.cc:194] Registered new tserver with Master: 2df4308c852240859ada53d216ab69ae (127.4.177.6:34269)
I20260812 06:36:12.047866  5814 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.4.177.6:39305
I20260812 06:36:12.142882  6118 heartbeater.cc:499] Master 127.4.177.62:42183 was elected leader, sending a full tablet report...
W20260812 06:36:12.156940  7182 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:36:12.157171  7182 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:36:12.157214  7182 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 06:36:12.160557  7182 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:36:12.160674  7182 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.177.7
I20260812 06:36:12.164695  7182 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-6/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.177.20:34335
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--consensus_rpc_timeout_ms=1000
--log_cache_size_limit_mb=10
--fs_data_dirs=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-6/data
--fs_wal_dir=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-6/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.177.7:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-6/data/info.pb
--webserver_interface=127.4.177.7
--webserver_port=0
--tserver_master_addrs=127.4.177.62:42183
--never_fsync=true
--heap_profile_path=/tmp/kudu.7182
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.177.7

Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 06:36:12.165809  7182 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:36:12.166993  7182 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 06:36:12.173382  7197 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:36:12.173378  7194 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:36:12.173403  7192 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:36:12.173691  7182 server_base.cc:1061] running on GCE node
I20260812 06:36:12.174149  7182 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:36:12.174803  7182 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:36:12.175957  7182 hybrid_clock.cc:648] HybridClock initialized: now 1786516572175938 us; error 34 us; skew 500 ppm
I20260812 06:36:12.177781  7182 webserver.cc:533] Webserver started at http://127.4.177.7:38897/ using document root <none> and password file <none>
I20260812 06:36:12.178413  7182 fs_manager.cc:362] Metadata directory not provided
I20260812 06:36:12.178470  7182 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:36:12.178696  7182 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:36:12.180303  7182 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-6/data/instance:
uuid: "b037d17b21f8422fa5e41b70eb4dcd08"
format_stamp: "Formatted at 2026-08-12 06:36:12 on dist-test-slave-jztv"
I20260812 06:36:12.180794  7182 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-6/wal/instance:
uuid: "b037d17b21f8422fa5e41b70eb4dcd08"
format_stamp: "Formatted at 2026-08-12 06:36:12 on dist-test-slave-jztv"
I20260812 06:36:12.184370  7182 fs_manager.cc:696] Time spent creating directory manager: real 0.003s	user 0.005s	sys 0.000s
I20260812 06:36:12.186599  7206 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 06:36:12.187538  7182 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.000s	sys 0.003s
I20260812 06:36:12.187680  7182 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-6/data,/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-6/wal
uuid: "b037d17b21f8422fa5e41b70eb4dcd08"
format_stamp: "Formatted at 2026-08-12 06:36:12 on dist-test-slave-jztv"
I20260812 06:36:12.187783  7182 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-6/wal
metadata directory: /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-6/wal
1 data directories: /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-6/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:36:12.208196  7182 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 06:36:12.209035  7182 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:36:12.209245  7182 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:36:12.209879  7182 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260812 06:36:12.211020  7182 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:36:12.211092  7182 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260812 06:36:12.211161  7182 ts_tablet_manager.cc:616] Registered 0 tablets
I20260812 06:36:12.211212  7182 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260812 06:36:12.223451  7182 rpc_server.cc:307] RPC server started. Bound to: 127.4.177.7:46299
I20260812 06:36:12.223482  7391 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.4.177.7:46299 every 8 connection(s)
I20260812 06:36:12.224450  7182 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-6/data/info.pb
I20260812 06:36:12.224874  4804 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskCnZu03/build/debug/bin/kudu as pid 7182
I20260812 06:36:12.224990  4804 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-6/wal/instance
I20260812 06:36:12.233636  7392 heartbeater.cc:344] Connected to a master server at 127.4.177.62:42183
I20260812 06:36:12.233951  7392 heartbeater.cc:461] Registering TS with master...
I20260812 06:36:12.234640  7392 heartbeater.cc:507] Master 127.4.177.62:42183 requested a full tablet report, sending...
I20260812 06:36:12.235836  5814 ts_manager.cc:194] Registered new tserver with Master: b037d17b21f8422fa5e41b70eb4dcd08 (127.4.177.7:46299)
I20260812 06:36:12.236403  5814 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.4.177.7:48635
I20260812 06:36:12.239763  4804 external_mini_cluster.cc:955] 7 TS(s) registered with all masters
I20260812 06:36:12.249209  5814 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:36526:
name: "TestTable"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 7
split_rows_range_bounds {
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
owner: "alice"
W20260812 06:36:12.251799  5814 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table TestTable in case of a server failure: 8 tablet servers would be needed, 7 are available. Consider bringing up more tablet servers.
I20260812 06:36:12.271004  6667 tablet_service.cc:1511] Processing CreateTablet for tablet 7065bb403e3047dd9a0853c49318ef1f (DEFAULT_TABLE table=TestTable [id=28e140a647694af68de9d1ab622f5e59]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 06:36:12.271906  6667 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7065bb403e3047dd9a0853c49318ef1f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:36:12.271775  6893 tablet_service.cc:1511] Processing CreateTablet for tablet 7065bb403e3047dd9a0853c49318ef1f (DEFAULT_TABLE table=TestTable [id=28e140a647694af68de9d1ab622f5e59]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 06:36:12.272703  6893 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7065bb403e3047dd9a0853c49318ef1f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:36:12.272677  7087 tablet_service.cc:1511] Processing CreateTablet for tablet 7065bb403e3047dd9a0853c49318ef1f (DEFAULT_TABLE table=TestTable [id=28e140a647694af68de9d1ab622f5e59]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 06:36:12.273526  7087 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7065bb403e3047dd9a0853c49318ef1f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:36:12.276948  7432 tablet_bootstrap.cc:492] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02: Bootstrap starting.
I20260812 06:36:12.280640  7432 tablet_bootstrap.cc:654] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02: Neither blocks nor log segments found. Creating new log.
I20260812 06:36:12.281651  7432 log.cc:826] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02: Log is configured to *not* fsync() on all Append() calls
I20260812 06:36:12.282473  7435 tablet_bootstrap.cc:492] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790: Bootstrap starting.
I20260812 06:36:12.283571  6450 tablet_service.cc:1511] Processing CreateTablet for tablet 7065bb403e3047dd9a0853c49318ef1f (DEFAULT_TABLE table=TestTable [id=28e140a647694af68de9d1ab622f5e59]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 06:36:12.284610  6450 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7065bb403e3047dd9a0853c49318ef1f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:36:12.285379  7432 tablet_bootstrap.cc:492] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02: No bootstrap required, opened a new log
I20260812 06:36:12.285564  7432 ts_tablet_manager.cc:1403] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02: Time spent bootstrapping tablet: real 0.009s	user 0.004s	sys 0.000s
I20260812 06:36:12.289202  7432 raft_consensus.cc:359] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df4308c852240859ada53d216ab69ae" member_type: VOTER last_known_addr { host: "127.4.177.6" port: 34269 } } peers { permanent_uuid: "42895a9bcba64aa484f6509605035d02" member_type: VOTER last_known_addr { host: "127.4.177.4" port: 38181 } } peers { permanent_uuid: "222b20de5a1145c6b8a11df8374e8790" member_type: VOTER last_known_addr { host: "127.4.177.5" port: 43521 } } peers { permanent_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 40897 } } peers { permanent_uuid: "8d90d61da62549208fe117b4384a469b" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 41539 } } peers { permanent_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" member_type: VOTER last_known_addr { host: "127.4.177.7" port: 46299 } } peers { permanent_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 41057 } }
I20260812 06:36:12.289527  7432 raft_consensus.cc:385] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:36:12.289644  7432 raft_consensus.cc:740] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 42895a9bcba64aa484f6509605035d02, State: Initialized, Role: FOLLOWER
I20260812 06:36:12.290728  7432 consensus_queue.cc:260] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df4308c852240859ada53d216ab69ae" member_type: VOTER last_known_addr { host: "127.4.177.6" port: 34269 } } peers { permanent_uuid: "42895a9bcba64aa484f6509605035d02" member_type: VOTER last_known_addr { host: "127.4.177.4" port: 38181 } } peers { permanent_uuid: "222b20de5a1145c6b8a11df8374e8790" member_type: VOTER last_known_addr { host: "127.4.177.5" port: 43521 } } peers { permanent_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 40897 } } peers { permanent_uuid: "8d90d61da62549208fe117b4384a469b" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 41539 } } peers { permanent_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" member_type: VOTER last_known_addr { host: "127.4.177.7" port: 46299 } } peers { permanent_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 41057 } }
I20260812 06:36:12.291771  7432 ts_tablet_manager.cc:1434] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260812 06:36:12.291800  7291 tablet_service.cc:1511] Processing CreateTablet for tablet 7065bb403e3047dd9a0853c49318ef1f (DEFAULT_TABLE table=TestTable [id=28e140a647694af68de9d1ab622f5e59]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 06:36:12.292372  6784 heartbeater.cc:499] Master 127.4.177.62:42183 was elected leader, sending a full tablet report...
I20260812 06:36:12.292768  7291 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7065bb403e3047dd9a0853c49318ef1f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:36:12.296135  7439 tablet_bootstrap.cc:492] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae: Bootstrap starting.
I20260812 06:36:12.297827  6020 tablet_service.cc:1511] Processing CreateTablet for tablet 7065bb403e3047dd9a0853c49318ef1f (DEFAULT_TABLE table=TestTable [id=28e140a647694af68de9d1ab622f5e59]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 06:36:12.298296  6213 tablet_service.cc:1511] Processing CreateTablet for tablet 7065bb403e3047dd9a0853c49318ef1f (DEFAULT_TABLE table=TestTable [id=28e140a647694af68de9d1ab622f5e59]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 06:36:12.298786  6020 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7065bb403e3047dd9a0853c49318ef1f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:36:12.299152  6213 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7065bb403e3047dd9a0853c49318ef1f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:36:12.299167  7435 tablet_bootstrap.cc:654] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790: Neither blocks nor log segments found. Creating new log.
I20260812 06:36:12.300145  7435 log.cc:826] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790: Log is configured to *not* fsync() on all Append() calls
I20260812 06:36:12.300875  7439 tablet_bootstrap.cc:654] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae: Neither blocks nor log segments found. Creating new log.
I20260812 06:36:12.301693  7439 log.cc:826] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae: Log is configured to *not* fsync() on all Append() calls
I20260812 06:36:12.303370  7435 tablet_bootstrap.cc:492] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790: No bootstrap required, opened a new log
I20260812 06:36:12.303578  7435 ts_tablet_manager.cc:1403] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790: Time spent bootstrapping tablet: real 0.021s	user 0.004s	sys 0.000s
I20260812 06:36:12.304440  7450 tablet_bootstrap.cc:492] T 7065bb403e3047dd9a0853c49318ef1f P ec6c8d818e744cdbb6b0d15b321010e0: Bootstrap starting.
I20260812 06:36:12.306519  7450 tablet_bootstrap.cc:654] T 7065bb403e3047dd9a0853c49318ef1f P ec6c8d818e744cdbb6b0d15b321010e0: Neither blocks nor log segments found. Creating new log.
I20260812 06:36:12.306753  7451 tablet_bootstrap.cc:492] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b: Bootstrap starting.
I20260812 06:36:12.306861  7443 tablet_bootstrap.cc:492] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e: Bootstrap starting.
I20260812 06:36:12.307557  7450 log.cc:826] T 7065bb403e3047dd9a0853c49318ef1f P ec6c8d818e744cdbb6b0d15b321010e0: Log is configured to *not* fsync() on all Append() calls
I20260812 06:36:12.306427  7435 raft_consensus.cc:359] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df4308c852240859ada53d216ab69ae" member_type: VOTER last_known_addr { host: "127.4.177.6" port: 34269 } } peers { permanent_uuid: "42895a9bcba64aa484f6509605035d02" member_type: VOTER last_known_addr { host: "127.4.177.4" port: 38181 } } peers { permanent_uuid: "222b20de5a1145c6b8a11df8374e8790" member_type: VOTER last_known_addr { host: "127.4.177.5" port: 43521 } } peers { permanent_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 40897 } } peers { permanent_uuid: "8d90d61da62549208fe117b4384a469b" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 41539 } } peers { permanent_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" member_type: VOTER last_known_addr { host: "127.4.177.7" port: 46299 } } peers { permanent_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 41057 } }
I20260812 06:36:12.308616  7435 raft_consensus.cc:385] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:36:12.308688  7435 raft_consensus.cc:740] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 222b20de5a1145c6b8a11df8374e8790, State: Initialized, Role: FOLLOWER
I20260812 06:36:12.309113  7451 tablet_bootstrap.cc:654] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b: Neither blocks nor log segments found. Creating new log.
I20260812 06:36:12.309165  7447 tablet_bootstrap.cc:492] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08: Bootstrap starting.
I20260812 06:36:12.309201  7435 consensus_queue.cc:260] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df4308c852240859ada53d216ab69ae" member_type: VOTER last_known_addr { host: "127.4.177.6" port: 34269 } } peers { permanent_uuid: "42895a9bcba64aa484f6509605035d02" member_type: VOTER last_known_addr { host: "127.4.177.4" port: 38181 } } peers { permanent_uuid: "222b20de5a1145c6b8a11df8374e8790" member_type: VOTER last_known_addr { host: "127.4.177.5" port: 43521 } } peers { permanent_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 40897 } } peers { permanent_uuid: "8d90d61da62549208fe117b4384a469b" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 41539 } } peers { permanent_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" member_type: VOTER last_known_addr { host: "127.4.177.7" port: 46299 } } peers { permanent_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 41057 } }
I20260812 06:36:12.309434  7450 tablet_bootstrap.cc:492] T 7065bb403e3047dd9a0853c49318ef1f P ec6c8d818e744cdbb6b0d15b321010e0: No bootstrap required, opened a new log
I20260812 06:36:12.309638  7450 ts_tablet_manager.cc:1403] T 7065bb403e3047dd9a0853c49318ef1f P ec6c8d818e744cdbb6b0d15b321010e0: Time spent bootstrapping tablet: real 0.005s	user 0.005s	sys 0.000s
I20260812 06:36:12.309927  7451 log.cc:826] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b: Log is configured to *not* fsync() on all Append() calls
I20260812 06:36:12.310153  7435 ts_tablet_manager.cc:1434] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260812 06:36:12.310499  6985 heartbeater.cc:499] Master 127.4.177.62:42183 was elected leader, sending a full tablet report...
I20260812 06:36:12.311225  7447 tablet_bootstrap.cc:654] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08: Neither blocks nor log segments found. Creating new log.
I20260812 06:36:12.311997  7447 log.cc:826] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08: Log is configured to *not* fsync() on all Append() calls
I20260812 06:36:12.312779  7443 tablet_bootstrap.cc:654] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e: Neither blocks nor log segments found. Creating new log.
I20260812 06:36:12.313556  7443 log.cc:826] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e: Log is configured to *not* fsync() on all Append() calls
I20260812 06:36:12.313606  7439 tablet_bootstrap.cc:492] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae: No bootstrap required, opened a new log
I20260812 06:36:12.313818  7439 ts_tablet_manager.cc:1403] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae: Time spent bootstrapping tablet: real 0.022s	user 0.000s	sys 0.004s
I20260812 06:36:12.313683  7450 raft_consensus.cc:359] T 7065bb403e3047dd9a0853c49318ef1f P ec6c8d818e744cdbb6b0d15b321010e0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df4308c852240859ada53d216ab69ae" member_type: VOTER last_known_addr { host: "127.4.177.6" port: 34269 } } peers { permanent_uuid: "42895a9bcba64aa484f6509605035d02" member_type: VOTER last_known_addr { host: "127.4.177.4" port: 38181 } } peers { permanent_uuid: "222b20de5a1145c6b8a11df8374e8790" member_type: VOTER last_known_addr { host: "127.4.177.5" port: 43521 } } peers { permanent_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 40897 } } peers { permanent_uuid: "8d90d61da62549208fe117b4384a469b" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 41539 } } peers { permanent_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" member_type: VOTER last_known_addr { host: "127.4.177.7" port: 46299 } } peers { permanent_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 41057 } }
I20260812 06:36:12.313920  7450 raft_consensus.cc:385] T 7065bb403e3047dd9a0853c49318ef1f P ec6c8d818e744cdbb6b0d15b321010e0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:36:12.314002  7450 raft_consensus.cc:740] T 7065bb403e3047dd9a0853c49318ef1f P ec6c8d818e744cdbb6b0d15b321010e0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec6c8d818e744cdbb6b0d15b321010e0, State: Initialized, Role: FOLLOWER
I20260812 06:36:12.314589  7450 consensus_queue.cc:260] T 7065bb403e3047dd9a0853c49318ef1f P ec6c8d818e744cdbb6b0d15b321010e0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df4308c852240859ada53d216ab69ae" member_type: VOTER last_known_addr { host: "127.4.177.6" port: 34269 } } peers { permanent_uuid: "42895a9bcba64aa484f6509605035d02" member_type: VOTER last_known_addr { host: "127.4.177.4" port: 38181 } } peers { permanent_uuid: "222b20de5a1145c6b8a11df8374e8790" member_type: VOTER last_known_addr { host: "127.4.177.5" port: 43521 } } peers { permanent_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 40897 } } peers { permanent_uuid: "8d90d61da62549208fe117b4384a469b" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 41539 } } peers { permanent_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" member_type: VOTER last_known_addr { host: "127.4.177.7" port: 46299 } } peers { permanent_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 41057 } }
I20260812 06:36:12.315534  7443 tablet_bootstrap.cc:492] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e: No bootstrap required, opened a new log
I20260812 06:36:12.315570  7450 ts_tablet_manager.cc:1434] T 7065bb403e3047dd9a0853c49318ef1f P ec6c8d818e744cdbb6b0d15b321010e0: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260812 06:36:12.315730  7443 ts_tablet_manager.cc:1403] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e: Time spent bootstrapping tablet: real 0.009s	user 0.005s	sys 0.000s
I20260812 06:36:12.315754  7447 tablet_bootstrap.cc:492] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08: No bootstrap required, opened a new log
I20260812 06:36:12.316032  7447 ts_tablet_manager.cc:1403] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08: Time spent bootstrapping tablet: real 0.007s	user 0.005s	sys 0.000s
I20260812 06:36:12.317030  7439 raft_consensus.cc:359] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df4308c852240859ada53d216ab69ae" member_type: VOTER last_known_addr { host: "127.4.177.6" port: 34269 } } peers { permanent_uuid: "42895a9bcba64aa484f6509605035d02" member_type: VOTER last_known_addr { host: "127.4.177.4" port: 38181 } } peers { permanent_uuid: "222b20de5a1145c6b8a11df8374e8790" member_type: VOTER last_known_addr { host: "127.4.177.5" port: 43521 } } peers { permanent_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 40897 } } peers { permanent_uuid: "8d90d61da62549208fe117b4384a469b" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 41539 } } peers { permanent_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" member_type: VOTER last_known_addr { host: "127.4.177.7" port: 46299 } } peers { permanent_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 41057 } }
I20260812 06:36:12.317262  7439 raft_consensus.cc:385] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:36:12.317324  7439 raft_consensus.cc:740] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2df4308c852240859ada53d216ab69ae, State: Initialized, Role: FOLLOWER
I20260812 06:36:12.317791  7439 consensus_queue.cc:260] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df4308c852240859ada53d216ab69ae" member_type: VOTER last_known_addr { host: "127.4.177.6" port: 34269 } } peers { permanent_uuid: "42895a9bcba64aa484f6509605035d02" member_type: VOTER last_known_addr { host: "127.4.177.4" port: 38181 } } peers { permanent_uuid: "222b20de5a1145c6b8a11df8374e8790" member_type: VOTER last_known_addr { host: "127.4.177.5" port: 43521 } } peers { permanent_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 40897 } } peers { permanent_uuid: "8d90d61da62549208fe117b4384a469b" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 41539 } } peers { permanent_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" member_type: VOTER last_known_addr { host: "127.4.177.7" port: 46299 } } peers { permanent_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 41057 } }
I20260812 06:36:12.318809  7439 ts_tablet_manager.cc:1434] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.002s
I20260812 06:36:12.318953  7443 raft_consensus.cc:359] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df4308c852240859ada53d216ab69ae" member_type: VOTER last_known_addr { host: "127.4.177.6" port: 34269 } } peers { permanent_uuid: "42895a9bcba64aa484f6509605035d02" member_type: VOTER last_known_addr { host: "127.4.177.4" port: 38181 } } peers { permanent_uuid: "222b20de5a1145c6b8a11df8374e8790" member_type: VOTER last_known_addr { host: "127.4.177.5" port: 43521 } } peers { permanent_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 40897 } } peers { permanent_uuid: "8d90d61da62549208fe117b4384a469b" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 41539 } } peers { permanent_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" member_type: VOTER last_known_addr { host: "127.4.177.7" port: 46299 } } peers { permanent_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 41057 } }
I20260812 06:36:12.319180  7443 raft_consensus.cc:385] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:36:12.319188  7178 heartbeater.cc:499] Master 127.4.177.62:42183 was elected leader, sending a full tablet report...
I20260812 06:36:12.319245  7443 raft_consensus.cc:740] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7962f587ccd842ad8a3f6ed9a5d7707e, State: Initialized, Role: FOLLOWER
I20260812 06:36:12.319730  7443 consensus_queue.cc:260] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df4308c852240859ada53d216ab69ae" member_type: VOTER last_known_addr { host: "127.4.177.6" port: 34269 } } peers { permanent_uuid: "42895a9bcba64aa484f6509605035d02" member_type: VOTER last_known_addr { host: "127.4.177.4" port: 38181 } } peers { permanent_uuid: "222b20de5a1145c6b8a11df8374e8790" member_type: VOTER last_known_addr { host: "127.4.177.5" port: 43521 } } peers { permanent_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 40897 } } peers { permanent_uuid: "8d90d61da62549208fe117b4384a469b" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 41539 } } peers { permanent_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" member_type: VOTER last_known_addr { host: "127.4.177.7" port: 46299 } } peers { permanent_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 41057 } }
I20260812 06:36:12.319655  7447 raft_consensus.cc:359] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df4308c852240859ada53d216ab69ae" member_type: VOTER last_known_addr { host: "127.4.177.6" port: 34269 } } peers { permanent_uuid: "42895a9bcba64aa484f6509605035d02" member_type: VOTER last_known_addr { host: "127.4.177.4" port: 38181 } } peers { permanent_uuid: "222b20de5a1145c6b8a11df8374e8790" member_type: VOTER last_known_addr { host: "127.4.177.5" port: 43521 } } peers { permanent_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 40897 } } peers { permanent_uuid: "8d90d61da62549208fe117b4384a469b" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 41539 } } peers { permanent_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" member_type: VOTER last_known_addr { host: "127.4.177.7" port: 46299 } } peers { permanent_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 41057 } }
I20260812 06:36:12.319959  7447 raft_consensus.cc:385] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:36:12.320127  7447 raft_consensus.cc:740] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b037d17b21f8422fa5e41b70eb4dcd08, State: Initialized, Role: FOLLOWER
I20260812 06:36:12.320634  7443 ts_tablet_manager.cc:1434] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260812 06:36:12.320601  7447 consensus_queue.cc:260] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df4308c852240859ada53d216ab69ae" member_type: VOTER last_known_addr { host: "127.4.177.6" port: 34269 } } peers { permanent_uuid: "42895a9bcba64aa484f6509605035d02" member_type: VOTER last_known_addr { host: "127.4.177.4" port: 38181 } } peers { permanent_uuid: "222b20de5a1145c6b8a11df8374e8790" member_type: VOTER last_known_addr { host: "127.4.177.5" port: 43521 } } peers { permanent_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 40897 } } peers { permanent_uuid: "8d90d61da62549208fe117b4384a469b" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 41539 } } peers { permanent_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" member_type: VOTER last_known_addr { host: "127.4.177.7" port: 46299 } } peers { permanent_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 41057 } }
I20260812 06:36:12.321183  6556 heartbeater.cc:499] Master 127.4.177.62:42183 was elected leader, sending a full tablet report...
I20260812 06:36:12.321400  7392 heartbeater.cc:499] Master 127.4.177.62:42183 was elected leader, sending a full tablet report...
I20260812 06:36:12.321712  7447 ts_tablet_manager.cc:1434] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08: Time spent starting tablet: real 0.006s	user 0.000s	sys 0.005s
I20260812 06:36:12.322970  7451 tablet_bootstrap.cc:492] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b: No bootstrap required, opened a new log
I20260812 06:36:12.323213  7451 ts_tablet_manager.cc:1403] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b: Time spent bootstrapping tablet: real 0.017s	user 0.003s	sys 0.001s
I20260812 06:36:12.326536  7451 raft_consensus.cc:359] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df4308c852240859ada53d216ab69ae" member_type: VOTER last_known_addr { host: "127.4.177.6" port: 34269 } } peers { permanent_uuid: "42895a9bcba64aa484f6509605035d02" member_type: VOTER last_known_addr { host: "127.4.177.4" port: 38181 } } peers { permanent_uuid: "222b20de5a1145c6b8a11df8374e8790" member_type: VOTER last_known_addr { host: "127.4.177.5" port: 43521 } } peers { permanent_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 40897 } } peers { permanent_uuid: "8d90d61da62549208fe117b4384a469b" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 41539 } } peers { permanent_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" member_type: VOTER last_known_addr { host: "127.4.177.7" port: 46299 } } peers { permanent_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 41057 } }
I20260812 06:36:12.326757  7451 raft_consensus.cc:385] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:36:12.326851  7451 raft_consensus.cc:740] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8d90d61da62549208fe117b4384a469b, State: Initialized, Role: FOLLOWER
I20260812 06:36:12.327319  7451 consensus_queue.cc:260] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df4308c852240859ada53d216ab69ae" member_type: VOTER last_known_addr { host: "127.4.177.6" port: 34269 } } peers { permanent_uuid: "42895a9bcba64aa484f6509605035d02" member_type: VOTER last_known_addr { host: "127.4.177.4" port: 38181 } } peers { permanent_uuid: "222b20de5a1145c6b8a11df8374e8790" member_type: VOTER last_known_addr { host: "127.4.177.5" port: 43521 } } peers { permanent_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 40897 } } peers { permanent_uuid: "8d90d61da62549208fe117b4384a469b" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 41539 } } peers { permanent_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" member_type: VOTER last_known_addr { host: "127.4.177.7" port: 46299 } } peers { permanent_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 41057 } }
I20260812 06:36:12.328083  6328 heartbeater.cc:499] Master 127.4.177.62:42183 was elected leader, sending a full tablet report...
I20260812 06:36:12.328461  7451 ts_tablet_manager.cc:1434] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.002s
W20260812 06:36:12.331910  6330 tablet.cc:2406] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 06:36:12.358585  6986 tablet.cc:2406] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 06:36:12.379853  6119 tablet.cc:2406] T 7065bb403e3047dd9a0853c49318ef1f P ec6c8d818e744cdbb6b0d15b321010e0: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 06:36:12.432798  6785 tablet.cc:2406] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 06:36:12.475698  7393 tablet.cc:2406] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 06:36:12.479173  7467 raft_consensus.cc:493] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 06:36:12.479581  7467 raft_consensus.cc:515] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df4308c852240859ada53d216ab69ae" member_type: VOTER last_known_addr { host: "127.4.177.6" port: 34269 } } peers { permanent_uuid: "42895a9bcba64aa484f6509605035d02" member_type: VOTER last_known_addr { host: "127.4.177.4" port: 38181 } } peers { permanent_uuid: "222b20de5a1145c6b8a11df8374e8790" member_type: VOTER last_known_addr { host: "127.4.177.5" port: 43521 } } peers { permanent_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 40897 } } peers { permanent_uuid: "8d90d61da62549208fe117b4384a469b" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 41539 } } peers { permanent_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" member_type: VOTER last_known_addr { host: "127.4.177.7" port: 46299 } } peers { permanent_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 41057 } }
I20260812 06:36:12.481097  7467 leader_election.cc:290] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2df4308c852240859ada53d216ab69ae (127.4.177.6:34269), 42895a9bcba64aa484f6509605035d02 (127.4.177.4:38181), 222b20de5a1145c6b8a11df8374e8790 (127.4.177.5:43521), 8d90d61da62549208fe117b4384a469b (127.4.177.2:41539), b037d17b21f8422fa5e41b70eb4dcd08 (127.4.177.7:46299), ec6c8d818e744cdbb6b0d15b321010e0 (127.4.177.1:41057)
I20260812 06:36:12.486811  7322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" is_pre_election: true
I20260812 06:36:12.487170  7322 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7962f587ccd842ad8a3f6ed9a5d7707e in term 0.
I20260812 06:36:12.487679  6703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "42895a9bcba64aa484f6509605035d02" is_pre_election: true
I20260812 06:36:12.487955  6703 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7962f587ccd842ad8a3f6ed9a5d7707e in term 0.
I20260812 06:36:12.487996  7109 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2df4308c852240859ada53d216ab69ae" is_pre_election: true
I20260812 06:36:12.488250  7109 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7962f587ccd842ad8a3f6ed9a5d7707e in term 0.
I20260812 06:36:12.488790  6369 leader_election.cc:304] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 7 voters: 4 yes votes; 0 no votes. yes voters: 2df4308c852240859ada53d216ab69ae, 42895a9bcba64aa484f6509605035d02, 7962f587ccd842ad8a3f6ed9a5d7707e, b037d17b21f8422fa5e41b70eb4dcd08; no voters: 
I20260812 06:36:12.489069  7467 raft_consensus.cc:2804] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 06:36:12.489169  7467 raft_consensus.cc:493] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 06:36:12.489248  7467 raft_consensus.cc:3060] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:36:12.489645  6921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "222b20de5a1145c6b8a11df8374e8790" is_pre_election: true
I20260812 06:36:12.489873  6921 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7962f587ccd842ad8a3f6ed9a5d7707e in term 0.
I20260812 06:36:12.490513  7467 raft_consensus.cc:515] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df4308c852240859ada53d216ab69ae" member_type: VOTER last_known_addr { host: "127.4.177.6" port: 34269 } } peers { permanent_uuid: "42895a9bcba64aa484f6509605035d02" member_type: VOTER last_known_addr { host: "127.4.177.4" port: 38181 } } peers { permanent_uuid: "222b20de5a1145c6b8a11df8374e8790" member_type: VOTER last_known_addr { host: "127.4.177.5" port: 43521 } } peers { permanent_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 40897 } } peers { permanent_uuid: "8d90d61da62549208fe117b4384a469b" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 41539 } } peers { permanent_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" member_type: VOTER last_known_addr { host: "127.4.177.7" port: 46299 } } peers { permanent_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 41057 } }
I20260812 06:36:12.491199  7467 leader_election.cc:290] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [CANDIDATE]: Term 1 election: Requested vote from peers 2df4308c852240859ada53d216ab69ae (127.4.177.6:34269), 42895a9bcba64aa484f6509605035d02 (127.4.177.4:38181), 222b20de5a1145c6b8a11df8374e8790 (127.4.177.5:43521), 8d90d61da62549208fe117b4384a469b (127.4.177.2:41539), b037d17b21f8422fa5e41b70eb4dcd08 (127.4.177.7:46299), ec6c8d818e744cdbb6b0d15b321010e0 (127.4.177.1:41057)
I20260812 06:36:12.491335  6249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8d90d61da62549208fe117b4384a469b" is_pre_election: true
I20260812 06:36:12.491580  6249 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7962f587ccd842ad8a3f6ed9a5d7707e in term 0.
I20260812 06:36:12.491571  7109 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2df4308c852240859ada53d216ab69ae"
I20260812 06:36:12.491616  6248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8d90d61da62549208fe117b4384a469b"
I20260812 06:36:12.491717  6248 raft_consensus.cc:3060] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:36:12.491708  7109 raft_consensus.cc:3060] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:36:12.492115  6703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "42895a9bcba64aa484f6509605035d02"
I20260812 06:36:12.492237  6703 raft_consensus.cc:3060] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:36:12.492257  7322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b037d17b21f8422fa5e41b70eb4dcd08"
I20260812 06:36:12.492364  7322 raft_consensus.cc:3060] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:36:12.492832  7109 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7962f587ccd842ad8a3f6ed9a5d7707e in term 1.
I20260812 06:36:12.492832  6248 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7962f587ccd842ad8a3f6ed9a5d7707e in term 1.
I20260812 06:36:12.493108  6921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "222b20de5a1145c6b8a11df8374e8790"
I20260812 06:36:12.493249  6921 raft_consensus.cc:3060] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:36:12.493381  6703 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7962f587ccd842ad8a3f6ed9a5d7707e in term 1.
I20260812 06:36:12.493381  7322 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7962f587ccd842ad8a3f6ed9a5d7707e in term 1.
I20260812 06:36:12.493818  6369 leader_election.cc:304] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 7 voters: 4 yes votes; 0 no votes. yes voters: 2df4308c852240859ada53d216ab69ae, 7962f587ccd842ad8a3f6ed9a5d7707e, 8d90d61da62549208fe117b4384a469b, b037d17b21f8422fa5e41b70eb4dcd08; no voters: 
I20260812 06:36:12.494030  7467 raft_consensus.cc:2804] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [term 1 FOLLOWER]: Leader election won for term 1
I20260812 06:36:12.494273  7467 raft_consensus.cc:697] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [term 1 LEADER]: Becoming Leader. State: Replica: 7962f587ccd842ad8a3f6ed9a5d7707e, State: Running, Role: LEADER
I20260812 06:36:12.494638  7467 consensus_queue.cc:237] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df4308c852240859ada53d216ab69ae" member_type: VOTER last_known_addr { host: "127.4.177.6" port: 34269 } } peers { permanent_uuid: "42895a9bcba64aa484f6509605035d02" member_type: VOTER last_known_addr { host: "127.4.177.4" port: 38181 } } peers { permanent_uuid: "222b20de5a1145c6b8a11df8374e8790" member_type: VOTER last_known_addr { host: "127.4.177.5" port: 43521 } } peers { permanent_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 40897 } } peers { permanent_uuid: "8d90d61da62549208fe117b4384a469b" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 41539 } } peers { permanent_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" member_type: VOTER last_known_addr { host: "127.4.177.7" port: 46299 } } peers { permanent_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 41057 } }
I20260812 06:36:12.494975  6921 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7962f587ccd842ad8a3f6ed9a5d7707e in term 1.
I20260812 06:36:12.495012  6047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec6c8d818e744cdbb6b0d15b321010e0"
I20260812 06:36:12.495088  6048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" is_pre_election: true
I20260812 06:36:12.495216  6047 raft_consensus.cc:3060] T 7065bb403e3047dd9a0853c49318ef1f P ec6c8d818e744cdbb6b0d15b321010e0 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:36:12.496336  6047 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P ec6c8d818e744cdbb6b0d15b321010e0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7962f587ccd842ad8a3f6ed9a5d7707e in term 1.
I20260812 06:36:12.498802  5810 catalog_manager.cc:5719] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e reported cstate change: term changed from 0 to 1, leader changed from <none> to 7962f587ccd842ad8a3f6ed9a5d7707e (127.4.177.3). New cstate: current_term: 1 leader_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df4308c852240859ada53d216ab69ae" member_type: VOTER last_known_addr { host: "127.4.177.6" port: 34269 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "42895a9bcba64aa484f6509605035d02" member_type: VOTER last_known_addr { host: "127.4.177.4" port: 38181 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "222b20de5a1145c6b8a11df8374e8790" member_type: VOTER last_known_addr { host: "127.4.177.5" port: 43521 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 40897 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8d90d61da62549208fe117b4384a469b" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 41539 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" member_type: VOTER last_known_addr { host: "127.4.177.7" port: 46299 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 41057 } health_report { overall_health: UNKNOWN } } }
W20260812 06:36:12.507485  6557 tablet.cc:2406] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 06:36:12.537978  7179 tablet.cc:2406] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 06:36:12.559324  4804 external_mini_cluster.cc:955] 7 TS(s) registered with all masters
I20260812 06:36:12.560835  4804 ts_itest-base.cc:246] Waiting for 1 tablets on tserver ec6c8d818e744cdbb6b0d15b321010e0 to finish bootstrapping
I20260812 06:36:12.569800  4804 ts_itest-base.cc:246] Waiting for 1 tablets on tserver 8d90d61da62549208fe117b4384a469b to finish bootstrapping
I20260812 06:36:12.575726  4804 ts_itest-base.cc:246] Waiting for 1 tablets on tserver 7962f587ccd842ad8a3f6ed9a5d7707e to finish bootstrapping
I20260812 06:36:12.582460  4804 ts_itest-base.cc:246] Waiting for 1 tablets on tserver 42895a9bcba64aa484f6509605035d02 to finish bootstrapping
I20260812 06:36:12.589746  4804 ts_itest-base.cc:246] Waiting for 1 tablets on tserver 222b20de5a1145c6b8a11df8374e8790 to finish bootstrapping
I20260812 06:36:12.596226  4804 ts_itest-base.cc:246] Waiting for 1 tablets on tserver 2df4308c852240859ada53d216ab69ae to finish bootstrapping
I20260812 06:36:12.601609  4804 ts_itest-base.cc:246] Waiting for 1 tablets on tserver b037d17b21f8422fa5e41b70eb4dcd08 to finish bootstrapping
I20260812 06:36:12.630961  6703 raft_consensus.cc:1275] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [term 1 FOLLOWER]: Refusing update from remote peer 7962f587ccd842ad8a3f6ed9a5d7707e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 06:36:12.630961  6921 raft_consensus.cc:1275] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [term 1 FOLLOWER]: Refusing update from remote peer 7962f587ccd842ad8a3f6ed9a5d7707e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 06:36:12.631211  6248 raft_consensus.cc:1275] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b [term 1 FOLLOWER]: Refusing update from remote peer 7962f587ccd842ad8a3f6ed9a5d7707e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 06:36:12.631915  7480 consensus_queue.cc:1048] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8d90d61da62549208fe117b4384a469b" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 41539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 06:36:12.632225  7504 consensus_queue.cc:1048] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [LEADER]: Connected to new peer: Peer: permanent_uuid: "42895a9bcba64aa484f6509605035d02" member_type: VOTER last_known_addr { host: "127.4.177.4" port: 38181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 06:36:12.632285  7322 raft_consensus.cc:1275] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08 [term 1 FOLLOWER]: Refusing update from remote peer 7962f587ccd842ad8a3f6ed9a5d7707e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 06:36:12.633021  6047 raft_consensus.cc:1275] T 7065bb403e3047dd9a0853c49318ef1f P ec6c8d818e744cdbb6b0d15b321010e0 [term 1 FOLLOWER]: Refusing update from remote peer 7962f587ccd842ad8a3f6ed9a5d7707e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 06:36:12.633749  7109 raft_consensus.cc:1275] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [term 1 FOLLOWER]: Refusing update from remote peer 7962f587ccd842ad8a3f6ed9a5d7707e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 06:36:12.634735  7480 consensus_queue.cc:1048] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [LEADER]: Connected to new peer: Peer: permanent_uuid: "2df4308c852240859ada53d216ab69ae" member_type: VOTER last_known_addr { host: "127.4.177.6" port: 34269 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 06:36:12.635121  7480 consensus_queue.cc:1048] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 41057 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 06:36:12.637929  7467 consensus_queue.cc:1048] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [LEADER]: Connected to new peer: Peer: permanent_uuid: "222b20de5a1145c6b8a11df8374e8790" member_type: VOTER last_known_addr { host: "127.4.177.5" port: 43521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 06:36:12.638301  7467 consensus_queue.cc:1048] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [LEADER]: Connected to new peer: Peer: permanent_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" member_type: VOTER last_known_addr { host: "127.4.177.7" port: 46299 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 06:36:12.651479  7506 mvcc.cc:204] Tried to move back new op lower bound from 7317571881497595904 to 7317571880943648768. Current Snapshot: MvccSnapshot[applied={T|T < 7317571881487228928}]
I20260812 06:36:12.655552  7507 mvcc.cc:204] Tried to move back new op lower bound from 7317571881507917824 to 7317571880943648768. Current Snapshot: MvccSnapshot[applied={T|T < 7317571881493164032}]
I20260812 06:36:12.667081  7512 mvcc.cc:204] Tried to move back new op lower bound from 7317571881507917824 to 7317571880943648768. Current Snapshot: MvccSnapshot[applied={T|T < 7317571881506168832}]
I20260812 06:36:12.671932  7518 mvcc.cc:204] Tried to move back new op lower bound from 7317571881507917824 to 7317571880943648768. Current Snapshot: MvccSnapshot[applied={T|T < 7317571881506168832}]
I20260812 06:36:12.735054  7534 mvcc.cc:204] Tried to move back new op lower bound from 7317571881507917824 to 7317571880943648768. Current Snapshot: MvccSnapshot[applied={T|T < 7317571881507917824 or (T in {7317571881507917824})}]
I20260812 06:36:15.285578  7596 raft_consensus.cc:493] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7962f587ccd842ad8a3f6ed9a5d7707e)
I20260812 06:36:15.285732  7596 raft_consensus.cc:515] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df4308c852240859ada53d216ab69ae" member_type: VOTER last_known_addr { host: "127.4.177.6" port: 34269 } } peers { permanent_uuid: "42895a9bcba64aa484f6509605035d02" member_type: VOTER last_known_addr { host: "127.4.177.4" port: 38181 } } peers { permanent_uuid: "222b20de5a1145c6b8a11df8374e8790" member_type: VOTER last_known_addr { host: "127.4.177.5" port: 43521 } } peers { permanent_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 40897 } } peers { permanent_uuid: "8d90d61da62549208fe117b4384a469b" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 41539 } } peers { permanent_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" member_type: VOTER last_known_addr { host: "127.4.177.7" port: 46299 } } peers { permanent_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 41057 } }
I20260812 06:36:15.287058  7596 leader_election.cc:290] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2df4308c852240859ada53d216ab69ae (127.4.177.6:34269), 42895a9bcba64aa484f6509605035d02 (127.4.177.4:38181), 222b20de5a1145c6b8a11df8374e8790 (127.4.177.5:43521), 7962f587ccd842ad8a3f6ed9a5d7707e (127.4.177.3:40897), b037d17b21f8422fa5e41b70eb4dcd08 (127.4.177.7:46299), ec6c8d818e744cdbb6b0d15b321010e0 (127.4.177.1:41057)
I20260812 06:36:15.293004  7322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "8d90d61da62549208fe117b4384a469b" candidate_term: 2 candidate_status { last_received { term: 1 index: 311 } } ignore_live_leader: false dest_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" is_pre_election: true
I20260812 06:36:15.294013  6919 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "8d90d61da62549208fe117b4384a469b" candidate_term: 2 candidate_status { last_received { term: 1 index: 311 } } ignore_live_leader: false dest_uuid: "222b20de5a1145c6b8a11df8374e8790" is_pre_election: true
I20260812 06:36:15.294153  6701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "8d90d61da62549208fe117b4384a469b" candidate_term: 2 candidate_status { last_received { term: 1 index: 311 } } ignore_live_leader: false dest_uuid: "42895a9bcba64aa484f6509605035d02" is_pre_election: true
I20260812 06:36:15.295097  6047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "8d90d61da62549208fe117b4384a469b" candidate_term: 2 candidate_status { last_received { term: 1 index: 311 } } ignore_live_leader: false dest_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" is_pre_election: true
I20260812 06:36:15.295275  7108 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "8d90d61da62549208fe117b4384a469b" candidate_term: 2 candidate_status { last_received { term: 1 index: 311 } } ignore_live_leader: false dest_uuid: "2df4308c852240859ada53d216ab69ae" is_pre_election: true
I20260812 06:36:15.295639  6150 leader_election.cc:304] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 5 responses out of 7 voters: 1 yes votes; 4 no votes. yes voters: 8d90d61da62549208fe117b4384a469b; no voters: 222b20de5a1145c6b8a11df8374e8790, 42895a9bcba64aa484f6509605035d02, b037d17b21f8422fa5e41b70eb4dcd08, ec6c8d818e744cdbb6b0d15b321010e0
I20260812 06:36:15.295921  7596 raft_consensus.cc:2749] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 06:36:15.404018  7613 raft_consensus.cc:493] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7962f587ccd842ad8a3f6ed9a5d7707e)
I20260812 06:36:15.404238  7613 raft_consensus.cc:515] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df4308c852240859ada53d216ab69ae" member_type: VOTER last_known_addr { host: "127.4.177.6" port: 34269 } } peers { permanent_uuid: "42895a9bcba64aa484f6509605035d02" member_type: VOTER last_known_addr { host: "127.4.177.4" port: 38181 } } peers { permanent_uuid: "222b20de5a1145c6b8a11df8374e8790" member_type: VOTER last_known_addr { host: "127.4.177.5" port: 43521 } } peers { permanent_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 40897 } } peers { permanent_uuid: "8d90d61da62549208fe117b4384a469b" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 41539 } } peers { permanent_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" member_type: VOTER last_known_addr { host: "127.4.177.7" port: 46299 } } peers { permanent_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 41057 } }
I20260812 06:36:15.405694  7613 leader_election.cc:290] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2df4308c852240859ada53d216ab69ae (127.4.177.6:34269), 222b20de5a1145c6b8a11df8374e8790 (127.4.177.5:43521), 7962f587ccd842ad8a3f6ed9a5d7707e (127.4.177.3:40897), 8d90d61da62549208fe117b4384a469b (127.4.177.2:41539), b037d17b21f8422fa5e41b70eb4dcd08 (127.4.177.7:46299), ec6c8d818e744cdbb6b0d15b321010e0 (127.4.177.1:41057)
I20260812 06:36:15.411026  7322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "42895a9bcba64aa484f6509605035d02" candidate_term: 2 candidate_status { last_received { term: 1 index: 363 } } ignore_live_leader: false dest_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" is_pre_election: true
I20260812 06:36:15.411128  7108 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "42895a9bcba64aa484f6509605035d02" candidate_term: 2 candidate_status { last_received { term: 1 index: 363 } } ignore_live_leader: false dest_uuid: "2df4308c852240859ada53d216ab69ae" is_pre_election: true
I20260812 06:36:15.411234  6919 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "42895a9bcba64aa484f6509605035d02" candidate_term: 2 candidate_status { last_received { term: 1 index: 363 } } ignore_live_leader: false dest_uuid: "222b20de5a1145c6b8a11df8374e8790" is_pre_election: true
I20260812 06:36:15.411286  7108 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 42895a9bcba64aa484f6509605035d02 in term 1.
I20260812 06:36:15.411679  7627 raft_consensus.cc:493] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7962f587ccd842ad8a3f6ed9a5d7707e)
I20260812 06:36:15.411865  7627 raft_consensus.cc:515] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df4308c852240859ada53d216ab69ae" member_type: VOTER last_known_addr { host: "127.4.177.6" port: 34269 } } peers { permanent_uuid: "42895a9bcba64aa484f6509605035d02" member_type: VOTER last_known_addr { host: "127.4.177.4" port: 38181 } } peers { permanent_uuid: "222b20de5a1145c6b8a11df8374e8790" member_type: VOTER last_known_addr { host: "127.4.177.5" port: 43521 } } peers { permanent_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 40897 } } peers { permanent_uuid: "8d90d61da62549208fe117b4384a469b" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 41539 } } peers { permanent_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" member_type: VOTER last_known_addr { host: "127.4.177.7" port: 46299 } } peers { permanent_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 41057 } }
I20260812 06:36:15.413475  7627 leader_election.cc:290] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 42895a9bcba64aa484f6509605035d02 (127.4.177.4:38181), 222b20de5a1145c6b8a11df8374e8790 (127.4.177.5:43521), 7962f587ccd842ad8a3f6ed9a5d7707e (127.4.177.3:40897), 8d90d61da62549208fe117b4384a469b (127.4.177.2:41539), b037d17b21f8422fa5e41b70eb4dcd08 (127.4.177.7:46299), ec6c8d818e744cdbb6b0d15b321010e0 (127.4.177.1:41057)
I20260812 06:36:15.413416  6248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "42895a9bcba64aa484f6509605035d02" candidate_term: 2 candidate_status { last_received { term: 1 index: 363 } } ignore_live_leader: false dest_uuid: "8d90d61da62549208fe117b4384a469b" is_pre_election: true
I20260812 06:36:15.413609  6248 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 42895a9bcba64aa484f6509605035d02 in term 1.
I20260812 06:36:15.415443  6047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "42895a9bcba64aa484f6509605035d02" candidate_term: 2 candidate_status { last_received { term: 1 index: 363 } } ignore_live_leader: false dest_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" is_pre_election: true
I20260812 06:36:15.418972  6919 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "2df4308c852240859ada53d216ab69ae" candidate_term: 2 candidate_status { last_received { term: 1 index: 363 } } ignore_live_leader: false dest_uuid: "222b20de5a1145c6b8a11df8374e8790" is_pre_election: true
I20260812 06:36:15.419000  6701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "2df4308c852240859ada53d216ab69ae" candidate_term: 2 candidate_status { last_received { term: 1 index: 363 } } ignore_live_leader: false dest_uuid: "42895a9bcba64aa484f6509605035d02" is_pre_election: true
I20260812 06:36:15.419123  6701 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2df4308c852240859ada53d216ab69ae in term 1.
I20260812 06:36:15.420393  7322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "2df4308c852240859ada53d216ab69ae" candidate_term: 2 candidate_status { last_received { term: 1 index: 363 } } ignore_live_leader: false dest_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" is_pre_election: true
I20260812 06:36:15.420442  6248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "2df4308c852240859ada53d216ab69ae" candidate_term: 2 candidate_status { last_received { term: 1 index: 363 } } ignore_live_leader: false dest_uuid: "8d90d61da62549208fe117b4384a469b" is_pre_election: true
I20260812 06:36:15.420598  6248 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2df4308c852240859ada53d216ab69ae in term 1.
I20260812 06:36:15.422729  6047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "2df4308c852240859ada53d216ab69ae" candidate_term: 2 candidate_status { last_received { term: 1 index: 363 } } ignore_live_leader: false dest_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" is_pre_election: true
I20260812 06:36:15.752035  7639 raft_consensus.cc:493] T 7065bb403e3047dd9a0853c49318ef1f P ec6c8d818e744cdbb6b0d15b321010e0 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7962f587ccd842ad8a3f6ed9a5d7707e)
I20260812 06:36:15.752256  7639 raft_consensus.cc:515] T 7065bb403e3047dd9a0853c49318ef1f P ec6c8d818e744cdbb6b0d15b321010e0 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df4308c852240859ada53d216ab69ae" member_type: VOTER last_known_addr { host: "127.4.177.6" port: 34269 } } peers { permanent_uuid: "42895a9bcba64aa484f6509605035d02" member_type: VOTER last_known_addr { host: "127.4.177.4" port: 38181 } } peers { permanent_uuid: "222b20de5a1145c6b8a11df8374e8790" member_type: VOTER last_known_addr { host: "127.4.177.5" port: 43521 } } peers { permanent_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 40897 } } peers { permanent_uuid: "8d90d61da62549208fe117b4384a469b" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 41539 } } peers { permanent_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" member_type: VOTER last_known_addr { host: "127.4.177.7" port: 46299 } } peers { permanent_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 41057 } }
I20260812 06:36:15.753826  7639 leader_election.cc:290] T 7065bb403e3047dd9a0853c49318ef1f P ec6c8d818e744cdbb6b0d15b321010e0 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2df4308c852240859ada53d216ab69ae (127.4.177.6:34269), 42895a9bcba64aa484f6509605035d02 (127.4.177.4:38181), 222b20de5a1145c6b8a11df8374e8790 (127.4.177.5:43521), 7962f587ccd842ad8a3f6ed9a5d7707e (127.4.177.3:40897), 8d90d61da62549208fe117b4384a469b (127.4.177.2:41539), b037d17b21f8422fa5e41b70eb4dcd08 (127.4.177.7:46299)
I20260812 06:36:15.758750  7322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" candidate_term: 2 candidate_status { last_received { term: 1 index: 241 } } ignore_live_leader: false dest_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" is_pre_election: true
I20260812 06:36:15.758955  7322 raft_consensus.cc:2410] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ec6c8d818e744cdbb6b0d15b321010e0 for term 2 because replica has last-logged OpId of term: 1 index: 279, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 241.
I20260812 06:36:15.759589  7108 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" candidate_term: 2 candidate_status { last_received { term: 1 index: 241 } } ignore_live_leader: false dest_uuid: "2df4308c852240859ada53d216ab69ae" is_pre_election: true
I20260812 06:36:15.759768  7108 raft_consensus.cc:2410] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ec6c8d818e744cdbb6b0d15b321010e0 for term 2 because replica has last-logged OpId of term: 1 index: 363, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 241.
I20260812 06:36:15.760288  7652 raft_consensus.cc:493] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7962f587ccd842ad8a3f6ed9a5d7707e)
I20260812 06:36:15.760449  7652 raft_consensus.cc:515] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df4308c852240859ada53d216ab69ae" member_type: VOTER last_known_addr { host: "127.4.177.6" port: 34269 } } peers { permanent_uuid: "42895a9bcba64aa484f6509605035d02" member_type: VOTER last_known_addr { host: "127.4.177.4" port: 38181 } } peers { permanent_uuid: "222b20de5a1145c6b8a11df8374e8790" member_type: VOTER last_known_addr { host: "127.4.177.5" port: 43521 } } peers { permanent_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 40897 } } peers { permanent_uuid: "8d90d61da62549208fe117b4384a469b" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 41539 } } peers { permanent_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" member_type: VOTER last_known_addr { host: "127.4.177.7" port: 46299 } } peers { permanent_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 41057 } }
I20260812 06:36:15.761801  6701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" candidate_term: 2 candidate_status { last_received { term: 1 index: 241 } } ignore_live_leader: false dest_uuid: "42895a9bcba64aa484f6509605035d02" is_pre_election: true
I20260812 06:36:15.761967  6701 raft_consensus.cc:2410] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ec6c8d818e744cdbb6b0d15b321010e0 for term 2 because replica has last-logged OpId of term: 1 index: 363, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 241.
I20260812 06:36:15.762269  6919 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" candidate_term: 2 candidate_status { last_received { term: 1 index: 241 } } ignore_live_leader: false dest_uuid: "222b20de5a1145c6b8a11df8374e8790" is_pre_election: true
I20260812 06:36:15.762291  7652 leader_election.cc:290] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2df4308c852240859ada53d216ab69ae (127.4.177.6:34269), 42895a9bcba64aa484f6509605035d02 (127.4.177.4:38181), 222b20de5a1145c6b8a11df8374e8790 (127.4.177.5:43521), 7962f587ccd842ad8a3f6ed9a5d7707e (127.4.177.3:40897), 8d90d61da62549208fe117b4384a469b (127.4.177.2:41539), ec6c8d818e744cdbb6b0d15b321010e0 (127.4.177.1:41057)
I20260812 06:36:15.762462  6919 raft_consensus.cc:2410] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ec6c8d818e744cdbb6b0d15b321010e0 for term 2 because replica has last-logged OpId of term: 1 index: 363, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 241.
I20260812 06:36:15.762830  5945 leader_election.cc:304] T 7065bb403e3047dd9a0853c49318ef1f P ec6c8d818e744cdbb6b0d15b321010e0 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 5 responses out of 7 voters: 1 yes votes; 4 no votes. yes voters: ec6c8d818e744cdbb6b0d15b321010e0; no voters: 222b20de5a1145c6b8a11df8374e8790, 2df4308c852240859ada53d216ab69ae, 42895a9bcba64aa484f6509605035d02, b037d17b21f8422fa5e41b70eb4dcd08
I20260812 06:36:15.762967  7649 raft_consensus.cc:493] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7962f587ccd842ad8a3f6ed9a5d7707e)
I20260812 06:36:15.763068  7649 raft_consensus.cc:515] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df4308c852240859ada53d216ab69ae" member_type: VOTER last_known_addr { host: "127.4.177.6" port: 34269 } } peers { permanent_uuid: "42895a9bcba64aa484f6509605035d02" member_type: VOTER last_known_addr { host: "127.4.177.4" port: 38181 } } peers { permanent_uuid: "222b20de5a1145c6b8a11df8374e8790" member_type: VOTER last_known_addr { host: "127.4.177.5" port: 43521 } } peers { permanent_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 40897 } } peers { permanent_uuid: "8d90d61da62549208fe117b4384a469b" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 41539 } } peers { permanent_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" member_type: VOTER last_known_addr { host: "127.4.177.7" port: 46299 } } peers { permanent_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 41057 } }
I20260812 06:36:15.764662  7649 leader_election.cc:290] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2df4308c852240859ada53d216ab69ae (127.4.177.6:34269), 42895a9bcba64aa484f6509605035d02 (127.4.177.4:38181), 7962f587ccd842ad8a3f6ed9a5d7707e (127.4.177.3:40897), 8d90d61da62549208fe117b4384a469b (127.4.177.2:41539), b037d17b21f8422fa5e41b70eb4dcd08 (127.4.177.7:46299), ec6c8d818e744cdbb6b0d15b321010e0 (127.4.177.1:41057)
I20260812 06:36:15.766083  7639 raft_consensus.cc:2749] T 7065bb403e3047dd9a0853c49318ef1f P ec6c8d818e744cdbb6b0d15b321010e0 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 06:36:15.766675  6248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" candidate_term: 2 candidate_status { last_received { term: 1 index: 241 } } ignore_live_leader: false dest_uuid: "8d90d61da62549208fe117b4384a469b" is_pre_election: true
I20260812 06:36:15.766871  6248 raft_consensus.cc:2410] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ec6c8d818e744cdbb6b0d15b321010e0 for term 2 because replica has last-logged OpId of term: 1 index: 311, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 241.
I20260812 06:36:15.770004  6701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" candidate_term: 2 candidate_status { last_received { term: 1 index: 279 } } ignore_live_leader: false dest_uuid: "42895a9bcba64aa484f6509605035d02" is_pre_election: true
I20260812 06:36:15.770004  6919 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" candidate_term: 2 candidate_status { last_received { term: 1 index: 279 } } ignore_live_leader: false dest_uuid: "222b20de5a1145c6b8a11df8374e8790" is_pre_election: true
I20260812 06:36:15.770175  6919 raft_consensus.cc:2410] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b037d17b21f8422fa5e41b70eb4dcd08 for term 2 because replica has last-logged OpId of term: 1 index: 363, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 279.
I20260812 06:36:15.770175  6701 raft_consensus.cc:2410] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b037d17b21f8422fa5e41b70eb4dcd08 for term 2 because replica has last-logged OpId of term: 1 index: 363, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 279.
I20260812 06:36:15.771174  6248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" candidate_term: 2 candidate_status { last_received { term: 1 index: 279 } } ignore_live_leader: false dest_uuid: "8d90d61da62549208fe117b4384a469b" is_pre_election: true
I20260812 06:36:15.771377  6248 raft_consensus.cc:2410] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b037d17b21f8422fa5e41b70eb4dcd08 for term 2 because replica has last-logged OpId of term: 1 index: 311, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 279.
I20260812 06:36:15.772349  6047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" candidate_term: 2 candidate_status { last_received { term: 1 index: 279 } } ignore_live_leader: false dest_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" is_pre_election: true
I20260812 06:36:15.772519  6047 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P ec6c8d818e744cdbb6b0d15b321010e0 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b037d17b21f8422fa5e41b70eb4dcd08 in term 1.
I20260812 06:36:15.774317  7108 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" candidate_term: 2 candidate_status { last_received { term: 1 index: 279 } } ignore_live_leader: false dest_uuid: "2df4308c852240859ada53d216ab69ae" is_pre_election: true
I20260812 06:36:15.774482  7108 raft_consensus.cc:2410] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b037d17b21f8422fa5e41b70eb4dcd08 for term 2 because replica has last-logged OpId of term: 1 index: 363, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 279.
I20260812 06:36:15.775000  7211 leader_election.cc:304] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 6 responses out of 7 voters: 2 yes votes; 4 no votes. yes voters: b037d17b21f8422fa5e41b70eb4dcd08, ec6c8d818e744cdbb6b0d15b321010e0; no voters: 222b20de5a1145c6b8a11df8374e8790, 2df4308c852240859ada53d216ab69ae, 42895a9bcba64aa484f6509605035d02, 8d90d61da62549208fe117b4384a469b
I20260812 06:36:15.775279  7652 raft_consensus.cc:2749] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 06:36:15.775851  7322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "222b20de5a1145c6b8a11df8374e8790" candidate_term: 2 candidate_status { last_received { term: 1 index: 363 } } ignore_live_leader: false dest_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" is_pre_election: true
I20260812 06:36:15.776003  7322 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 222b20de5a1145c6b8a11df8374e8790 in term 1.
I20260812 06:36:15.777091  6248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "222b20de5a1145c6b8a11df8374e8790" candidate_term: 2 candidate_status { last_received { term: 1 index: 363 } } ignore_live_leader: false dest_uuid: "8d90d61da62549208fe117b4384a469b" is_pre_election: true
I20260812 06:36:15.777241  6248 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 222b20de5a1145c6b8a11df8374e8790 in term 1.
I20260812 06:36:15.777921  7108 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "222b20de5a1145c6b8a11df8374e8790" candidate_term: 2 candidate_status { last_received { term: 1 index: 363 } } ignore_live_leader: false dest_uuid: "2df4308c852240859ada53d216ab69ae" is_pre_election: true
I20260812 06:36:15.778085  7108 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 222b20de5a1145c6b8a11df8374e8790 in term 1.
I20260812 06:36:15.778434  6047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "222b20de5a1145c6b8a11df8374e8790" candidate_term: 2 candidate_status { last_received { term: 1 index: 363 } } ignore_live_leader: false dest_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" is_pre_election: true
I20260812 06:36:15.778578  6047 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P ec6c8d818e744cdbb6b0d15b321010e0 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 222b20de5a1145c6b8a11df8374e8790 in term 1.
I20260812 06:36:15.778537  6821 leader_election.cc:304] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 7 voters: 4 yes votes; 0 no votes. yes voters: 222b20de5a1145c6b8a11df8374e8790, 2df4308c852240859ada53d216ab69ae, 8d90d61da62549208fe117b4384a469b, b037d17b21f8422fa5e41b70eb4dcd08; no voters: 
I20260812 06:36:15.778816  7649 raft_consensus.cc:2804] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260812 06:36:15.778879  7649 raft_consensus.cc:493] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 7962f587ccd842ad8a3f6ed9a5d7707e)
I20260812 06:36:15.778919  7649 raft_consensus.cc:3060] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [term 1 FOLLOWER]: Advancing to term 2
I20260812 06:36:15.780037  6701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "222b20de5a1145c6b8a11df8374e8790" candidate_term: 2 candidate_status { last_received { term: 1 index: 363 } } ignore_live_leader: false dest_uuid: "42895a9bcba64aa484f6509605035d02" is_pre_election: true
I20260812 06:36:15.780037  7649 raft_consensus.cc:515] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df4308c852240859ada53d216ab69ae" member_type: VOTER last_known_addr { host: "127.4.177.6" port: 34269 } } peers { permanent_uuid: "42895a9bcba64aa484f6509605035d02" member_type: VOTER last_known_addr { host: "127.4.177.4" port: 38181 } } peers { permanent_uuid: "222b20de5a1145c6b8a11df8374e8790" member_type: VOTER last_known_addr { host: "127.4.177.5" port: 43521 } } peers { permanent_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 40897 } } peers { permanent_uuid: "8d90d61da62549208fe117b4384a469b" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 41539 } } peers { permanent_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" member_type: VOTER last_known_addr { host: "127.4.177.7" port: 46299 } } peers { permanent_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 41057 } }
I20260812 06:36:15.780197  6701 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 222b20de5a1145c6b8a11df8374e8790 in term 1.
I20260812 06:36:15.780668  7649 leader_election.cc:290] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [CANDIDATE]: Term 2 election: Requested vote from peers 2df4308c852240859ada53d216ab69ae (127.4.177.6:34269), 42895a9bcba64aa484f6509605035d02 (127.4.177.4:38181), 7962f587ccd842ad8a3f6ed9a5d7707e (127.4.177.3:40897), 8d90d61da62549208fe117b4384a469b (127.4.177.2:41539), b037d17b21f8422fa5e41b70eb4dcd08 (127.4.177.7:46299), ec6c8d818e744cdbb6b0d15b321010e0 (127.4.177.1:41057)
I20260812 06:36:15.780857  7108 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "222b20de5a1145c6b8a11df8374e8790" candidate_term: 2 candidate_status { last_received { term: 1 index: 363 } } ignore_live_leader: false dest_uuid: "2df4308c852240859ada53d216ab69ae"
I20260812 06:36:15.780968  7108 raft_consensus.cc:3060] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [term 1 FOLLOWER]: Advancing to term 2
I20260812 06:36:15.780977  7322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "222b20de5a1145c6b8a11df8374e8790" candidate_term: 2 candidate_status { last_received { term: 1 index: 363 } } ignore_live_leader: false dest_uuid: "b037d17b21f8422fa5e41b70eb4dcd08"
I20260812 06:36:15.781101  7322 raft_consensus.cc:3060] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08 [term 1 FOLLOWER]: Advancing to term 2
I20260812 06:36:15.781093  6248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "222b20de5a1145c6b8a11df8374e8790" candidate_term: 2 candidate_status { last_received { term: 1 index: 363 } } ignore_live_leader: false dest_uuid: "8d90d61da62549208fe117b4384a469b"
I20260812 06:36:15.781208  6248 raft_consensus.cc:3060] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b [term 1 FOLLOWER]: Advancing to term 2
I20260812 06:36:15.781193  6701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "222b20de5a1145c6b8a11df8374e8790" candidate_term: 2 candidate_status { last_received { term: 1 index: 363 } } ignore_live_leader: false dest_uuid: "42895a9bcba64aa484f6509605035d02"
I20260812 06:36:15.781333  6701 raft_consensus.cc:3060] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [term 1 FOLLOWER]: Advancing to term 2
I20260812 06:36:15.782423  7322 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 222b20de5a1145c6b8a11df8374e8790 in term 2.
I20260812 06:36:15.782399  7108 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 222b20de5a1145c6b8a11df8374e8790 in term 2.
I20260812 06:36:15.782486  6248 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 222b20de5a1145c6b8a11df8374e8790 in term 2.
I20260812 06:36:15.782583  6701 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 222b20de5a1145c6b8a11df8374e8790 in term 2.
I20260812 06:36:15.782949  6821 leader_election.cc:304] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 7 voters: 4 yes votes; 0 no votes. yes voters: 222b20de5a1145c6b8a11df8374e8790, 42895a9bcba64aa484f6509605035d02, 8d90d61da62549208fe117b4384a469b, b037d17b21f8422fa5e41b70eb4dcd08; no voters: 
I20260812 06:36:15.783121  6047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "222b20de5a1145c6b8a11df8374e8790" candidate_term: 2 candidate_status { last_received { term: 1 index: 363 } } ignore_live_leader: false dest_uuid: "ec6c8d818e744cdbb6b0d15b321010e0"
I20260812 06:36:15.783183  7649 raft_consensus.cc:2804] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [term 2 FOLLOWER]: Leader election won for term 2
I20260812 06:36:15.783265  6047 raft_consensus.cc:3060] T 7065bb403e3047dd9a0853c49318ef1f P ec6c8d818e744cdbb6b0d15b321010e0 [term 1 FOLLOWER]: Advancing to term 2
I20260812 06:36:15.783453  7649 raft_consensus.cc:697] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [term 2 LEADER]: Becoming Leader. State: Replica: 222b20de5a1145c6b8a11df8374e8790, State: Running, Role: LEADER
I20260812 06:36:15.783812  7649 consensus_queue.cc:237] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 355, Committed index: 355, Last appended: 1.363, Last appended by leader: 363, Current term: 2, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df4308c852240859ada53d216ab69ae" member_type: VOTER last_known_addr { host: "127.4.177.6" port: 34269 } } peers { permanent_uuid: "42895a9bcba64aa484f6509605035d02" member_type: VOTER last_known_addr { host: "127.4.177.4" port: 38181 } } peers { permanent_uuid: "222b20de5a1145c6b8a11df8374e8790" member_type: VOTER last_known_addr { host: "127.4.177.5" port: 43521 } } peers { permanent_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 40897 } } peers { permanent_uuid: "8d90d61da62549208fe117b4384a469b" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 41539 } } peers { permanent_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" member_type: VOTER last_known_addr { host: "127.4.177.7" port: 46299 } } peers { permanent_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 41057 } }
I20260812 06:36:15.784304  6047 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P ec6c8d818e744cdbb6b0d15b321010e0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 222b20de5a1145c6b8a11df8374e8790 in term 2.
I20260812 06:36:15.786953  5810 catalog_manager.cc:5719] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 reported cstate change: term changed from 1 to 2, leader changed from 7962f587ccd842ad8a3f6ed9a5d7707e (127.4.177.3) to 222b20de5a1145c6b8a11df8374e8790 (127.4.177.5). New cstate: current_term: 2 leader_uuid: "222b20de5a1145c6b8a11df8374e8790" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df4308c852240859ada53d216ab69ae" member_type: VOTER last_known_addr { host: "127.4.177.6" port: 34269 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "42895a9bcba64aa484f6509605035d02" member_type: VOTER last_known_addr { host: "127.4.177.4" port: 38181 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "222b20de5a1145c6b8a11df8374e8790" member_type: VOTER last_known_addr { host: "127.4.177.5" port: 43521 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 40897 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8d90d61da62549208fe117b4384a469b" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 41539 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" member_type: VOTER last_known_addr { host: "127.4.177.7" port: 46299 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 41057 } health_report { overall_health: UNKNOWN } } }
W20260812 06:36:16.114416  6369 connection.cc:256] RPC call timeout handler was delayed by -1.641280725132674s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1s for a response
W20260812 06:36:16.114645  6369 connection.cc:256] RPC call timeout handler was delayed by -1.4302321781869978s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1s for a response
W20260812 06:36:16.114691  6369 connection.cc:256] RPC call timeout handler was delayed by -1.3080810422543436s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1s for a response
W20260812 06:36:16.118474  6364 connection.cc:256] RPC call timeout handler was delayed by -1.5238804600667208s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1s for a response
I20260812 06:36:16.121688  6701 raft_consensus.cc:1240] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [term 2 FOLLOWER]: Rejecting Update request from peer 7962f587ccd842ad8a3f6ed9a5d7707e for earlier term 1. Current term is 2. Ops: []
I20260812 06:36:16.121835  6248 raft_consensus.cc:1240] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b [term 2 FOLLOWER]: Rejecting Update request from peer 7962f587ccd842ad8a3f6ed9a5d7707e for earlier term 1. Current term is 2. Ops: [1.312-1.363]
I20260812 06:36:16.122253  6479 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "8d90d61da62549208fe117b4384a469b" candidate_term: 2 candidate_status { last_received { term: 1 index: 311 } } ignore_live_leader: false dest_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" is_pre_election: true
I20260812 06:36:16.123055  6047 raft_consensus.cc:1240] T 7065bb403e3047dd9a0853c49318ef1f P ec6c8d818e744cdbb6b0d15b321010e0 [term 2 FOLLOWER]: Rejecting Update request from peer 7962f587ccd842ad8a3f6ed9a5d7707e for earlier term 1. Current term is 2. Ops: [1.242-1.363]
I20260812 06:36:16.123776  7108 raft_consensus.cc:1240] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [term 2 FOLLOWER]: Rejecting Update request from peer 7962f587ccd842ad8a3f6ed9a5d7707e for earlier term 1. Current term is 2. Ops: []
I20260812 06:36:16.123762  7707 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:33412 (ReqId={client: 3067f3aef2a44cc4bb7f2ccd6acf83db, seq_no=354, attempt_no=0}) took 2229 ms. Trace:
I20260812 06:36:16.123848  7707 rpcz_store.cc:276] 0812 06:36:13.894162 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 06:36:13.894253 (+    91us) service_pool.cc:224] Handling call
0812 06:36:16.123728 (+2229475us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0812 06:36:13.895787 (+     0us) write_op.cc:183] PREPARE: starting on tablet 7065bb403e3047dd9a0853c49318ef1f
0812 06:36:13.895886 (+    99us) write_op.cc:432] Acquiring schema lock in shared mode
0812 06:36:13.895890 (+     4us) write_op.cc:435] Acquired schema lock
0812 06:36:13.895891 (+     1us) tablet.cc:662] Decoding operations
0812 06:36:13.895987 (+    96us) write_op.cc:620] Acquiring the partition lock for write op
0812 06:36:13.895993 (+     6us) write_op.cc:641] Partition lock acquired for write op
0812 06:36:13.895997 (+     4us) tablet.cc:685] Acquiring locks for 10 operations
0812 06:36:13.896060 (+    63us) tablet.cc:701] Row locks acquired
0812 06:36:13.896062 (+     2us) write_op.cc:260] PREPARE: finished
0812 06:36:13.896093 (+    31us) write_op.cc:270] Start()
0812 06:36:13.896112 (+    19us) write_op.cc:276] Timestamp: P: 1786516573896042 usec, L: 0
0812 06:36:13.896114 (+     2us) op_driver.cc:348] REPLICATION: starting
0812 06:36:13.896240 (+   126us) log.cc:844] Serialized 696 byte log entry
0812 06:36:16.118348 (+2222108us) op_driver.cc:464] REPLICATION: finished
0812 06:36:16.122919 (+  4571us) write_op.cc:301] APPLY: starting
0812 06:36:16.122938 (+    19us) tablet.cc:1367] starting BulkCheckPresence
0812 06:36:16.122982 (+    44us) tablet.cc:1370] finished BulkCheckPresence
0812 06:36:16.122996 (+    14us) tablet.cc:1372] starting ApplyRowOperation cycle
0812 06:36:16.123194 (+   198us) tablet.cc:1383] finished ApplyRowOperation cycle
0812 06:36:16.123227 (+    33us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0812 06:36:16.123232 (+     5us) write_op.cc:312] APPLY: finished
0812 06:36:16.123310 (+    78us) log.cc:844] Serialized 77 byte log entry
0812 06:36:16.123525 (+   215us) write_op.cc:489] Releasing partition, row and schema locks
0812 06:36:16.123627 (+   102us) write_op.cc:454] Released schema lock
0812 06:36:16.123636 (+     9us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":4505,"num_ops":10,"prepare.queue_time_us":1211,"prepare.run_cpu_time_us":517,"prepare.run_wall_time_us":516,"replication_time_us":2222197,"thread_start_us":207,"threads_started":2}]]}
I20260812 06:36:16.124995  6919 raft_consensus.cc:1240] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [term 2 LEADER]: Rejecting Update request from peer 7962f587ccd842ad8a3f6ed9a5d7707e for earlier term 1. Current term is 2. Ops: []
I20260812 06:36:16.125304  7707 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:33412 (ReqId={client: 3067f3aef2a44cc4bb7f2ccd6acf83db, seq_no=355, attempt_no=0}) took 2228 ms. Trace:
I20260812 06:36:16.125362  7707 rpcz_store.cc:276] 0812 06:36:13.896527 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 06:36:13.896600 (+    73us) service_pool.cc:224] Handling call
0812 06:36:16.125293 (+2228693us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0812 06:36:13.896968 (+     0us) write_op.cc:183] PREPARE: starting on tablet 7065bb403e3047dd9a0853c49318ef1f
0812 06:36:13.897050 (+    82us) write_op.cc:432] Acquiring schema lock in shared mode
0812 06:36:13.897053 (+     3us) write_op.cc:435] Acquired schema lock
0812 06:36:13.897054 (+     1us) tablet.cc:662] Decoding operations
0812 06:36:13.897133 (+    79us) write_op.cc:620] Acquiring the partition lock for write op
0812 06:36:13.897139 (+     6us) write_op.cc:641] Partition lock acquired for write op
0812 06:36:13.897142 (+     3us) tablet.cc:685] Acquiring locks for 10 operations
0812 06:36:13.897209 (+    67us) tablet.cc:701] Row locks acquired
0812 06:36:13.897211 (+     2us) write_op.cc:260] PREPARE: finished
0812 06:36:13.897234 (+    23us) write_op.cc:270] Start()
0812 06:36:13.897252 (+    18us) write_op.cc:276] Timestamp: P: 1786516573897185 usec, L: 0
0812 06:36:13.897254 (+     2us) op_driver.cc:348] REPLICATION: starting
0812 06:36:13.897354 (+   100us) log.cc:844] Serialized 696 byte log entry
0812 06:36:16.118496 (+2221142us) op_driver.cc:464] REPLICATION: finished
0812 06:36:16.124818 (+  6322us) write_op.cc:301] APPLY: starting
0812 06:36:16.124831 (+    13us) tablet.cc:1367] starting BulkCheckPresence
0812 06:36:16.124866 (+    35us) tablet.cc:1370] finished BulkCheckPresence
0812 06:36:16.124878 (+    12us) tablet.cc:1372] starting ApplyRowOperation cycle
0812 06:36:16.125072 (+   194us) tablet.cc:1383] finished ApplyRowOperation cycle
0812 06:36:16.125103 (+    31us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0812 06:36:16.125107 (+     4us) write_op.cc:312] APPLY: finished
0812 06:36:16.125174 (+    67us) log.cc:844] Serialized 77 byte log entry
0812 06:36:16.125217 (+    43us) write_op.cc:489] Releasing partition, row and schema locks
0812 06:36:16.125231 (+    14us) write_op.cc:454] Released schema lock
0812 06:36:16.125236 (+     5us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":6294,"num_ops":10,"prepare.queue_time_us":69,"prepare.run_cpu_time_us":434,"prepare.run_wall_time_us":433,"replication_time_us":2221228,"thread_start_us":43,"threads_started":1}]]}
I20260812 06:36:16.126056  7467 consensus_queue.cc:1059] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 41057 }, Status: INVALID_TERM, Last received: 1.241, Next index: 242, Last known committed idx: 233, Time since last communication: 0.000s
I20260812 06:36:16.126137  7707 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:33412 (ReqId={client: 3067f3aef2a44cc4bb7f2ccd6acf83db, seq_no=356, attempt_no=0}) took 2229 ms. Trace:
I20260812 06:36:16.126307  7322 raft_consensus.cc:1240] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08 [term 2 FOLLOWER]: Rejecting Update request from peer 7962f587ccd842ad8a3f6ed9a5d7707e for earlier term 1. Current term is 2. Ops: [1.280-1.363]
I20260812 06:36:16.126315  7574 consensus_queue.cc:1059] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "222b20de5a1145c6b8a11df8374e8790" member_type: VOTER last_known_addr { host: "127.4.177.5" port: 43521 }, Status: INVALID_TERM, Last received: 1.363, Next index: 364, Last known committed idx: 355, Time since last communication: 0.000s
I20260812 06:36:16.126194  7707 rpcz_store.cc:276] 0812 06:36:13.896653 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 06:36:13.896696 (+    43us) service_pool.cc:224] Handling call
0812 06:36:16.126124 (+2229428us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0812 06:36:13.897427 (+     0us) write_op.cc:183] PREPARE: starting on tablet 7065bb403e3047dd9a0853c49318ef1f
0812 06:36:13.897500 (+    73us) write_op.cc:432] Acquiring schema lock in shared mode
0812 06:36:13.897503 (+     3us) write_op.cc:435] Acquired schema lock
0812 06:36:13.897504 (+     1us) tablet.cc:662] Decoding operations
0812 06:36:13.897581 (+    77us) write_op.cc:620] Acquiring the partition lock for write op
0812 06:36:13.897588 (+     7us) write_op.cc:641] Partition lock acquired for write op
0812 06:36:13.897594 (+     6us) tablet.cc:685] Acquiring locks for 10 operations
0812 06:36:13.897655 (+    61us) tablet.cc:701] Row locks acquired
0812 06:36:13.897656 (+     1us) write_op.cc:260] PREPARE: finished
0812 06:36:13.897716 (+    60us) write_op.cc:270] Start()
0812 06:36:13.897734 (+    18us) write_op.cc:276] Timestamp: P: 1786516573897667 usec, L: 0
0812 06:36:13.897736 (+     2us) op_driver.cc:348] REPLICATION: starting
0812 06:36:13.897831 (+    95us) log.cc:844] Serialized 696 byte log entry
0812 06:36:16.118585 (+2220754us) op_driver.cc:464] REPLICATION: finished
0812 06:36:16.125715 (+  7130us) write_op.cc:301] APPLY: starting
0812 06:36:16.125727 (+    12us) tablet.cc:1367] starting BulkCheckPresence
0812 06:36:16.125756 (+    29us) tablet.cc:1370] finished BulkCheckPresence
0812 06:36:16.125767 (+    11us) tablet.cc:1372] starting ApplyRowOperation cycle
0812 06:36:16.125923 (+   156us) tablet.cc:1383] finished ApplyRowOperation cycle
0812 06:36:16.125952 (+    29us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0812 06:36:16.125957 (+     5us) write_op.cc:312] APPLY: finished
0812 06:36:16.126008 (+    51us) log.cc:844] Serialized 77 byte log entry
0812 06:36:16.126055 (+    47us) write_op.cc:489] Releasing partition, row and schema locks
0812 06:36:16.126069 (+    14us) write_op.cc:454] Released schema lock
0812 06:36:16.126073 (+     4us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"spinlock_wait_cycles":8064,"child_traces":[["op",{"apply.queue_time_us":7104,"mutex_wait_us":39,"num_ops":10,"prepare.queue_time_us":526,"prepare.run_cpu_time_us":467,"prepare.run_wall_time_us":467,"replication_time_us":2220836,"spinlock_wait_cycles":65280,"thread_start_us":52,"threads_started":1}]]}
I20260812 06:36:16.126423  7574 consensus_queue.cc:1243] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [LEADER]: Peer 222b20de5a1145c6b8a11df8374e8790 log is divergent from this leader: its last log entry 2.364 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 355
I20260812 06:36:16.126490  7574 consensus_queue.cc:1059] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8d90d61da62549208fe117b4384a469b" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 41539 }, Status: INVALID_TERM, Last received: 1.311, Next index: 312, Last known committed idx: 307, Time since last communication: 0.000s
I20260812 06:36:16.126574  7574 raft_consensus.cc:3055] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [term 1 LEADER]: Stepping down as leader of term 1
I20260812 06:36:16.126606  7574 raft_consensus.cc:740] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 7962f587ccd842ad8a3f6ed9a5d7707e, State: Running, Role: LEADER
I20260812 06:36:16.126762  7574 consensus_queue.cc:260] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 363, Committed index: 363, Last appended: 1.363, Last appended by leader: 363, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df4308c852240859ada53d216ab69ae" member_type: VOTER last_known_addr { host: "127.4.177.6" port: 34269 } } peers { permanent_uuid: "42895a9bcba64aa484f6509605035d02" member_type: VOTER last_known_addr { host: "127.4.177.4" port: 38181 } } peers { permanent_uuid: "222b20de5a1145c6b8a11df8374e8790" member_type: VOTER last_known_addr { host: "127.4.177.5" port: 43521 } } peers { permanent_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 40897 } } peers { permanent_uuid: "8d90d61da62549208fe117b4384a469b" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 41539 } } peers { permanent_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" member_type: VOTER last_known_addr { host: "127.4.177.7" port: 46299 } } peers { permanent_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 41057 } }
I20260812 06:36:16.126909  7707 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:33412 (ReqId={client: 3067f3aef2a44cc4bb7f2ccd6acf83db, seq_no=357, attempt_no=0}) took 2229 ms. Trace:
I20260812 06:36:16.126880  7503 consensus_queue.cc:1059] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "42895a9bcba64aa484f6509605035d02" member_type: VOTER last_known_addr { host: "127.4.177.4" port: 38181 }, Status: INVALID_TERM, Last received: 1.363, Next index: 364, Last known committed idx: 355, Time since last communication: 0.000s
I20260812 06:36:16.126950  7707 rpcz_store.cc:276] 0812 06:36:13.896988 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 06:36:13.897022 (+    34us) service_pool.cc:224] Handling call
0812 06:36:16.126902 (+2229880us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0812 06:36:13.897920 (+     0us) write_op.cc:183] PREPARE: starting on tablet 7065bb403e3047dd9a0853c49318ef1f
0812 06:36:13.897993 (+    73us) write_op.cc:432] Acquiring schema lock in shared mode
0812 06:36:13.897996 (+     3us) write_op.cc:435] Acquired schema lock
0812 06:36:13.897997 (+     1us) tablet.cc:662] Decoding operations
0812 06:36:13.898075 (+    78us) write_op.cc:620] Acquiring the partition lock for write op
0812 06:36:13.898080 (+     5us) write_op.cc:641] Partition lock acquired for write op
0812 06:36:13.898083 (+     3us) tablet.cc:685] Acquiring locks for 10 operations
0812 06:36:13.898150 (+    67us) tablet.cc:701] Row locks acquired
0812 06:36:13.898152 (+     2us) write_op.cc:260] PREPARE: finished
0812 06:36:13.898173 (+    21us) write_op.cc:270] Start()
0812 06:36:13.898191 (+    18us) write_op.cc:276] Timestamp: P: 1786516573898125 usec, L: 0
0812 06:36:13.898192 (+     1us) op_driver.cc:348] REPLICATION: starting
0812 06:36:13.898314 (+   122us) log.cc:844] Serialized 696 byte log entry
0812 06:36:16.118673 (+2220359us) op_driver.cc:464] REPLICATION: finished
0812 06:36:16.126528 (+  7855us) write_op.cc:301] APPLY: starting
0812 06:36:16.126538 (+    10us) tablet.cc:1367] starting BulkCheckPresence
0812 06:36:16.126565 (+    27us) tablet.cc:1370] finished BulkCheckPresence
0812 06:36:16.126576 (+    11us) tablet.cc:1372] starting ApplyRowOperation cycle
0812 06:36:16.126716 (+   140us) tablet.cc:1383] finished ApplyRowOperation cycle
0812 06:36:16.126748 (+    32us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0812 06:36:16.126753 (+     5us) write_op.cc:312] APPLY: finished
0812 06:36:16.126806 (+    53us) log.cc:844] Serialized 77 byte log entry
0812 06:36:16.126836 (+    30us) write_op.cc:489] Releasing partition, row and schema locks
0812 06:36:16.126847 (+    11us) write_op.cc:454] Released schema lock
0812 06:36:16.126851 (+     4us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":7831,"num_ops":10,"prepare.queue_time_us":662,"prepare.run_cpu_time_us":449,"prepare.run_wall_time_us":1771,"replication_time_us":2220471,"thread_start_us":70,"threads_started":1}]]}
I20260812 06:36:16.127650  7707 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:33412 (ReqId={client: 3067f3aef2a44cc4bb7f2ccd6acf83db, seq_no=358, attempt_no=0}) took 2230 ms. Trace:
W20260812 06:36:16.127453  7467 consensus_queue.cc:1175] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "2df4308c852240859ada53d216ab69ae" responder_term: 2 status { last_received { term: 1 index: 363 } last_committed_idx: 355 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 7962f587ccd842ad8a3f6ed9a5d7707e for earlier term 1. Current term is 2. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260812 06:36:16.128289  7703 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:33412 (ReqId={client: 3067f3aef2a44cc4bb7f2ccd6acf83db, seq_no=359, attempt_no=0}) took 2230 ms. Trace:
I20260812 06:36:16.129241  7703 rpcz_store.cc:276] 0812 06:36:13.897945 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 06:36:13.897978 (+    33us) service_pool.cc:224] Handling call
0812 06:36:16.128265 (+2230287us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0812 06:36:13.900345 (+     0us) write_op.cc:183] PREPARE: starting on tablet 7065bb403e3047dd9a0853c49318ef1f
0812 06:36:13.900412 (+    67us) write_op.cc:432] Acquiring schema lock in shared mode
0812 06:36:13.900415 (+     3us) write_op.cc:435] Acquired schema lock
0812 06:36:13.900417 (+     2us) tablet.cc:662] Decoding operations
0812 06:36:13.900498 (+    81us) write_op.cc:620] Acquiring the partition lock for write op
0812 06:36:13.900504 (+     6us) write_op.cc:641] Partition lock acquired for write op
0812 06:36:13.900506 (+     2us) tablet.cc:685] Acquiring locks for 10 operations
0812 06:36:13.900562 (+    56us) tablet.cc:701] Row locks acquired
0812 06:36:13.900563 (+     1us) write_op.cc:260] PREPARE: finished
0812 06:36:13.900590 (+    27us) write_op.cc:270] Start()
0812 06:36:13.900607 (+    17us) write_op.cc:276] Timestamp: P: 1786516573900541 usec, L: 0
0812 06:36:13.900608 (+     1us) op_driver.cc:348] REPLICATION: starting
0812 06:36:13.900704 (+    96us) log.cc:844] Serialized 696 byte log entry
0812 06:36:16.118800 (+2218096us) op_driver.cc:464] REPLICATION: finished
0812 06:36:16.127876 (+  9076us) write_op.cc:301] APPLY: starting
0812 06:36:16.127888 (+    12us) tablet.cc:1367] starting BulkCheckPresence
0812 06:36:16.127917 (+    29us) tablet.cc:1370] finished BulkCheckPresence
0812 06:36:16.127929 (+    12us) tablet.cc:1372] starting ApplyRowOperation cycle
0812 06:36:16.128094 (+   165us) tablet.cc:1383] finished ApplyRowOperation cycle
0812 06:36:16.128119 (+    25us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0812 06:36:16.128123 (+     4us) write_op.cc:312] APPLY: finished
0812 06:36:16.128173 (+    50us) log.cc:844] Serialized 77 byte log entry
0812 06:36:16.128205 (+    32us) write_op.cc:489] Releasing partition, row and schema locks
0812 06:36:16.128215 (+    10us) write_op.cc:454] Released schema lock
0812 06:36:16.128220 (+     5us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":9037,"num_ops":10,"prepare.queue_time_us":2137,"prepare.run_cpu_time_us":407,"prepare.run_wall_time_us":602,"replication_time_us":2218182}]]}
I20260812 06:36:16.130316  7703 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:33412 (ReqId={client: 3067f3aef2a44cc4bb7f2ccd6acf83db, seq_no=361, attempt_no=0}) took 2231 ms. Trace:
I20260812 06:36:16.130432  7703 rpcz_store.cc:276] 0812 06:36:13.898673 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 06:36:13.898704 (+    31us) service_pool.cc:224] Handling call
0812 06:36:16.130306 (+2231602us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0812 06:36:13.901592 (+     0us) write_op.cc:183] PREPARE: starting on tablet 7065bb403e3047dd9a0853c49318ef1f
0812 06:36:13.901663 (+    71us) write_op.cc:432] Acquiring schema lock in shared mode
0812 06:36:13.901668 (+     5us) write_op.cc:435] Acquired schema lock
0812 06:36:13.901669 (+     1us) tablet.cc:662] Decoding operations
0812 06:36:13.901756 (+    87us) write_op.cc:620] Acquiring the partition lock for write op
0812 06:36:13.901761 (+     5us) write_op.cc:641] Partition lock acquired for write op
0812 06:36:13.901764 (+     3us) tablet.cc:685] Acquiring locks for 10 operations
0812 06:36:13.901823 (+    59us) tablet.cc:701] Row locks acquired
0812 06:36:13.901826 (+     3us) write_op.cc:260] PREPARE: finished
0812 06:36:13.901869 (+    43us) write_op.cc:270] Start()
0812 06:36:13.901891 (+    22us) write_op.cc:276] Timestamp: P: 1786516573901817 usec, L: 0
0812 06:36:13.901892 (+     1us) op_driver.cc:348] REPLICATION: starting
0812 06:36:13.901999 (+   107us) log.cc:844] Serialized 676 byte log entry
0812 06:36:16.118847 (+2216848us) op_driver.cc:464] REPLICATION: finished
0812 06:36:16.129847 (+ 11000us) write_op.cc:301] APPLY: starting
0812 06:36:16.129857 (+    10us) tablet.cc:1367] starting BulkCheckPresence
0812 06:36:16.129886 (+    29us) tablet.cc:1370] finished BulkCheckPresence
0812 06:36:16.129897 (+    11us) tablet.cc:1372] starting ApplyRowOperation cycle
0812 06:36:16.130094 (+   197us) tablet.cc:1383] finished ApplyRowOperation cycle
0812 06:36:16.130122 (+    28us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0812 06:36:16.130126 (+     4us) write_op.cc:312] APPLY: finished
0812 06:36:16.130178 (+    52us) log.cc:844] Serialized 77 byte log entry
0812 06:36:16.130235 (+    57us) write_op.cc:489] Releasing partition, row and schema locks
0812 06:36:16.130249 (+    14us) write_op.cc:454] Released schema lock
0812 06:36:16.130254 (+     5us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":10973,"num_ops":10,"prepare.queue_time_us":2675,"prepare.run_cpu_time_us":487,"prepare.run_wall_time_us":487,"replication_time_us":2216947}]]}
I20260812 06:36:16.128861  7702 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:33412 (ReqId={client: 3067f3aef2a44cc4bb7f2ccd6acf83db, seq_no=360, attempt_no=0}) took 2230 ms. Trace:
I20260812 06:36:16.131052  7702 rpcz_store.cc:276] 0812 06:36:13.898329 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 06:36:13.898377 (+    48us) service_pool.cc:224] Handling call
0812 06:36:16.128837 (+2230460us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0812 06:36:13.900971 (+     0us) write_op.cc:183] PREPARE: starting on tablet 7065bb403e3047dd9a0853c49318ef1f
0812 06:36:13.901035 (+    64us) write_op.cc:432] Acquiring schema lock in shared mode
0812 06:36:13.901038 (+     3us) write_op.cc:435] Acquired schema lock
0812 06:36:13.901039 (+     1us) tablet.cc:662] Decoding operations
0812 06:36:13.901116 (+    77us) write_op.cc:620] Acquiring the partition lock for write op
0812 06:36:13.901122 (+     6us) write_op.cc:641] Partition lock acquired for write op
0812 06:36:13.901124 (+     2us) tablet.cc:685] Acquiring locks for 10 operations
0812 06:36:13.901183 (+    59us) tablet.cc:701] Row locks acquired
0812 06:36:13.901186 (+     3us) write_op.cc:260] PREPARE: finished
0812 06:36:13.901215 (+    29us) write_op.cc:270] Start()
0812 06:36:13.901232 (+    17us) write_op.cc:276] Timestamp: P: 1786516573901165 usec, L: 0
0812 06:36:13.901234 (+     2us) op_driver.cc:348] REPLICATION: starting
0812 06:36:13.901341 (+   107us) log.cc:844] Serialized 686 byte log entry
0812 06:36:16.118827 (+2217486us) op_driver.cc:464] REPLICATION: finished
0812 06:36:16.128415 (+  9588us) write_op.cc:301] APPLY: starting
0812 06:36:16.128425 (+    10us) tablet.cc:1367] starting BulkCheckPresence
0812 06:36:16.128451 (+    26us) tablet.cc:1370] finished BulkCheckPresence
0812 06:36:16.128466 (+    15us) tablet.cc:1372] starting ApplyRowOperation cycle
0812 06:36:16.128650 (+   184us) tablet.cc:1383] finished ApplyRowOperation cycle
0812 06:36:16.128678 (+    28us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0812 06:36:16.128683 (+     5us) write_op.cc:312] APPLY: finished
0812 06:36:16.128741 (+    58us) log.cc:844] Serialized 77 byte log entry
0812 06:36:16.128774 (+    33us) write_op.cc:489] Releasing partition, row and schema locks
0812 06:36:16.128785 (+    11us) write_op.cc:454] Released schema lock
0812 06:36:16.128790 (+     5us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":9554,"num_ops":10,"prepare.queue_time_us":2374,"prepare.run_cpu_time_us":418,"prepare.run_wall_time_us":589,"replication_time_us":2217583}]]}
I20260812 06:36:16.131655  6480 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "2df4308c852240859ada53d216ab69ae" candidate_term: 2 candidate_status { last_received { term: 1 index: 363 } } ignore_live_leader: false dest_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" is_pre_election: true
I20260812 06:36:16.131615  7707 rpcz_store.cc:276] 0812 06:36:13.897355 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 06:36:13.897618 (+   263us) service_pool.cc:224] Handling call
0812 06:36:16.127639 (+2230021us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0812 06:36:13.899723 (+     0us) write_op.cc:183] PREPARE: starting on tablet 7065bb403e3047dd9a0853c49318ef1f
0812 06:36:13.899785 (+    62us) write_op.cc:432] Acquiring schema lock in shared mode
0812 06:36:13.899788 (+     3us) write_op.cc:435] Acquired schema lock
0812 06:36:13.899790 (+     2us) tablet.cc:662] Decoding operations
0812 06:36:13.899872 (+    82us) write_op.cc:620] Acquiring the partition lock for write op
0812 06:36:13.899876 (+     4us) write_op.cc:641] Partition lock acquired for write op
0812 06:36:13.899879 (+     3us) tablet.cc:685] Acquiring locks for 10 operations
0812 06:36:13.899932 (+    53us) tablet.cc:701] Row locks acquired
0812 06:36:13.899934 (+     2us) write_op.cc:260] PREPARE: finished
0812 06:36:13.899958 (+    24us) write_op.cc:270] Start()
0812 06:36:13.899976 (+    18us) write_op.cc:276] Timestamp: P: 1786516573899910 usec, L: 0
0812 06:36:13.899978 (+     2us) op_driver.cc:348] REPLICATION: starting
0812 06:36:13.900071 (+    93us) log.cc:844] Serialized 696 byte log entry
0812 06:36:16.118775 (+2218704us) op_driver.cc:464] REPLICATION: finished
0812 06:36:16.127276 (+  8501us) write_op.cc:301] APPLY: starting
0812 06:36:16.127284 (+     8us) tablet.cc:1367] starting BulkCheckPresence
0812 06:36:16.127309 (+    25us) tablet.cc:1370] finished BulkCheckPresence
0812 06:36:16.127319 (+    10us) tablet.cc:1372] starting ApplyRowOperation cycle
0812 06:36:16.127469 (+   150us) tablet.cc:1383] finished ApplyRowOperation cycle
0812 06:36:16.127493 (+    24us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0812 06:36:16.127497 (+     4us) write_op.cc:312] APPLY: finished
0812 06:36:16.127546 (+    49us) log.cc:844] Serialized 77 byte log entry
0812 06:36:16.127578 (+    32us) write_op.cc:489] Releasing partition, row and schema locks
0812 06:36:16.127589 (+    11us) write_op.cc:454] Released schema lock
0812 06:36:16.127593 (+     4us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":8475,"num_ops":10,"prepare.queue_time_us":1868,"prepare.run_cpu_time_us":396,"prepare.run_wall_time_us":598,"replication_time_us":2218786}]]}
I20260812 06:36:16.127658  7574 raft_consensus.cc:3060] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [term 1 FOLLOWER]: Advancing to term 2
I20260812 06:36:16.134375  6472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "222b20de5a1145c6b8a11df8374e8790" candidate_term: 2 candidate_status { last_received { term: 1 index: 363 } } ignore_live_leader: false dest_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e"
I20260812 06:36:16.134881  6479 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "222b20de5a1145c6b8a11df8374e8790" candidate_term: 2 candidate_status { last_received { term: 1 index: 363 } } ignore_live_leader: false dest_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" is_pre_election: true
I20260812 06:36:16.135588  6480 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2df4308c852240859ada53d216ab69ae in term 2.
W20260812 06:36:16.136271  6403 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:33412: Illegal state: replica 7962f587ccd842ad8a3f6ed9a5d7707e is not leader of this config: current role FOLLOWER
W20260812 06:36:16.136358  6407 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:33412: Illegal state: replica 7962f587ccd842ad8a3f6ed9a5d7707e is not leader of this config: current role FOLLOWER
W20260812 06:36:16.136628  6406 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:33412: Illegal state: replica 7962f587ccd842ad8a3f6ed9a5d7707e is not leader of this config: current role FOLLOWER
W20260812 06:36:16.136874  6414 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:33412: Illegal state: replica 7962f587ccd842ad8a3f6ed9a5d7707e is not leader of this config: current role FOLLOWER
W20260812 06:36:16.136628  6411 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:33412: Illegal state: replica 7962f587ccd842ad8a3f6ed9a5d7707e is not leader of this config: current role FOLLOWER
I20260812 06:36:16.138316  7020 leader_election.cc:304] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 7 responses out of 7 voters: 4 yes votes; 3 no votes. yes voters: 2df4308c852240859ada53d216ab69ae, 42895a9bcba64aa484f6509605035d02, 7962f587ccd842ad8a3f6ed9a5d7707e, 8d90d61da62549208fe117b4384a469b; no voters: 222b20de5a1145c6b8a11df8374e8790, b037d17b21f8422fa5e41b70eb4dcd08, ec6c8d818e744cdbb6b0d15b321010e0
W20260812 06:36:16.138417  6404 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:33412: Illegal state: replica 7962f587ccd842ad8a3f6ed9a5d7707e is not leader of this config: current role FOLLOWER
W20260812 06:36:16.138803  6407 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:33412: Illegal state: replica 7962f587ccd842ad8a3f6ed9a5d7707e is not leader of this config: current role FOLLOWER
I20260812 06:36:16.138882  6479 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" candidate_term: 2 candidate_status { last_received { term: 1 index: 279 } } ignore_live_leader: false dest_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" is_pre_election: true
I20260812 06:36:16.139019  6479 raft_consensus.cc:2410] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b037d17b21f8422fa5e41b70eb4dcd08 for term 2 because replica has last-logged OpId of term: 1 index: 363, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 279.
W20260812 06:36:16.138803  6406 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:33412: Illegal state: replica 7962f587ccd842ad8a3f6ed9a5d7707e is not leader of this config: current role FOLLOWER
I20260812 06:36:16.139220  6480 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" candidate_term: 2 candidate_status { last_received { term: 1 index: 241 } } ignore_live_leader: false dest_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" is_pre_election: true
I20260812 06:36:16.139379  6480 raft_consensus.cc:2410] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ec6c8d818e744cdbb6b0d15b321010e0 for term 2 because replica has last-logged OpId of term: 1 index: 363, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 241.
I20260812 06:36:16.140014  6480 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "42895a9bcba64aa484f6509605035d02" candidate_term: 2 candidate_status { last_received { term: 1 index: 363 } } ignore_live_leader: false dest_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" is_pre_election: true
I20260812 06:36:16.140127  6480 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 42895a9bcba64aa484f6509605035d02 in term 2.
I20260812 06:36:16.140378  7714 raft_consensus.cc:2764] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [term 2 FOLLOWER]: Leader pre-election decision vote started in defunct term 1: won
I20260812 06:36:16.140568  6597 leader_election.cc:304] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 7 responses out of 7 voters: 4 yes votes; 3 no votes. yes voters: 2df4308c852240859ada53d216ab69ae, 42895a9bcba64aa484f6509605035d02, 7962f587ccd842ad8a3f6ed9a5d7707e, 8d90d61da62549208fe117b4384a469b; no voters: 222b20de5a1145c6b8a11df8374e8790, b037d17b21f8422fa5e41b70eb4dcd08, ec6c8d818e744cdbb6b0d15b321010e0
I20260812 06:36:16.141232  7716 raft_consensus.cc:2764] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [term 2 FOLLOWER]: Leader pre-election decision vote started in defunct term 1: won
W20260812 06:36:16.146147  7061 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50492: Illegal state: replica 2df4308c852240859ada53d216ab69ae is not leader of this config: current role FOLLOWER
W20260812 06:36:16.146147  7063 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50492: Illegal state: replica 2df4308c852240859ada53d216ab69ae is not leader of this config: current role FOLLOWER
W20260812 06:36:16.146503  7049 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50492: Illegal state: replica 2df4308c852240859ada53d216ab69ae is not leader of this config: current role FOLLOWER
W20260812 06:36:16.146548  7054 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50492: Illegal state: replica 2df4308c852240859ada53d216ab69ae is not leader of this config: current role FOLLOWER
W20260812 06:36:16.146948  7059 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50492: Illegal state: replica 2df4308c852240859ada53d216ab69ae is not leader of this config: current role FOLLOWER
W20260812 06:36:16.146147  7060 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50492: Illegal state: replica 2df4308c852240859ada53d216ab69ae is not leader of this config: current role FOLLOWER
W20260812 06:36:16.147064  7050 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50492: Illegal state: replica 2df4308c852240859ada53d216ab69ae is not leader of this config: current role FOLLOWER
W20260812 06:36:16.147065  7051 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50492: Illegal state: replica 2df4308c852240859ada53d216ab69ae is not leader of this config: current role FOLLOWER
W20260812 06:36:16.155012  6631 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:42408: Illegal state: replica 42895a9bcba64aa484f6509605035d02 is not leader of this config: current role FOLLOWER
W20260812 06:36:16.155088  6620 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:42408: Illegal state: replica 42895a9bcba64aa484f6509605035d02 is not leader of this config: current role FOLLOWER
W20260812 06:36:16.155186  6621 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:42408: Illegal state: replica 42895a9bcba64aa484f6509605035d02 is not leader of this config: current role FOLLOWER
W20260812 06:36:16.155258  6626 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:42408: Illegal state: replica 42895a9bcba64aa484f6509605035d02 is not leader of this config: current role FOLLOWER
W20260812 06:36:16.155004  6628 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:42408: Illegal state: replica 42895a9bcba64aa484f6509605035d02 is not leader of this config: current role FOLLOWER
W20260812 06:36:16.155738  6625 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:42408: Illegal state: replica 42895a9bcba64aa484f6509605035d02 is not leader of this config: current role FOLLOWER
W20260812 06:36:16.155738  6623 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:42408: Illegal state: replica 42895a9bcba64aa484f6509605035d02 is not leader of this config: current role FOLLOWER
W20260812 06:36:16.154994  6629 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:42408: Illegal state: replica 42895a9bcba64aa484f6509605035d02 is not leader of this config: current role FOLLOWER
I20260812 06:36:16.166776  7322 raft_consensus.cc:1275] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08 [term 2 FOLLOWER]: Refusing update from remote peer 222b20de5a1145c6b8a11df8374e8790: Log matching property violated. Preceding OpId in replica: term: 1 index: 279. Preceding OpId from leader: term: 2 index: 365. (index mismatch)
I20260812 06:36:16.166865  6047 raft_consensus.cc:1275] T 7065bb403e3047dd9a0853c49318ef1f P ec6c8d818e744cdbb6b0d15b321010e0 [term 2 FOLLOWER]: Refusing update from remote peer 222b20de5a1145c6b8a11df8374e8790: Log matching property violated. Preceding OpId in replica: term: 1 index: 241. Preceding OpId from leader: term: 2 index: 365. (index mismatch)
I20260812 06:36:16.167158  6248 raft_consensus.cc:1275] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b [term 2 FOLLOWER]: Refusing update from remote peer 222b20de5a1145c6b8a11df8374e8790: Log matching property violated. Preceding OpId in replica: term: 1 index: 311. Preceding OpId from leader: term: 2 index: 365. (index mismatch)
I20260812 06:36:16.167305  7108 raft_consensus.cc:1275] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [term 2 FOLLOWER]: Refusing update from remote peer 222b20de5a1145c6b8a11df8374e8790: Log matching property violated. Preceding OpId in replica: term: 1 index: 363. Preceding OpId from leader: term: 2 index: 365. (index mismatch)
I20260812 06:36:16.167440  6701 raft_consensus.cc:1275] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [term 2 FOLLOWER]: Refusing update from remote peer 222b20de5a1145c6b8a11df8374e8790: Log matching property violated. Preceding OpId in replica: term: 1 index: 363. Preceding OpId from leader: term: 2 index: 365. (index mismatch)
I20260812 06:36:16.167497  7687 consensus_queue.cc:1048] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 41057 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 364, Last known committed idx: 233, Time since last communication: 0.000s
I20260812 06:36:16.167785  7730 consensus_queue.cc:1048] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2df4308c852240859ada53d216ab69ae" member_type: VOTER last_known_addr { host: "127.4.177.6" port: 34269 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 364, Last known committed idx: 355, Time since last communication: 0.000s
I20260812 06:36:16.168087  6480 raft_consensus.cc:1275] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [term 2 FOLLOWER]: Refusing update from remote peer 222b20de5a1145c6b8a11df8374e8790: Log matching property violated. Preceding OpId in replica: term: 1 index: 363. Preceding OpId from leader: term: 2 index: 365. (index mismatch)
I20260812 06:36:16.168246  7719 consensus_queue.cc:1048] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8d90d61da62549208fe117b4384a469b" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 41539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 364, Last known committed idx: 307, Time since last communication: 0.000s
I20260812 06:36:16.169818  7730 consensus_queue.cc:1048] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [LEADER]: Connected to new peer: Peer: permanent_uuid: "42895a9bcba64aa484f6509605035d02" member_type: VOTER last_known_addr { host: "127.4.177.4" port: 38181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 364, Last known committed idx: 355, Time since last communication: 0.000s
I20260812 06:36:16.170202  7725 consensus_queue.cc:1048] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 40897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 364, Last known committed idx: 363, Time since last communication: 0.000s
I20260812 06:36:16.170394  7649 consensus_queue.cc:1048] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" member_type: VOTER last_known_addr { host: "127.4.177.7" port: 46299 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 364, Last known committed idx: 271, Time since last communication: 0.000s
I20260812 06:36:16.185926  7718 mvcc.cc:204] Tried to move back new op lower bound from 7317571895987036160 to 7317571894415798272. Current Snapshot: MvccSnapshot[applied={T|T < 7317571886678188032}]
I20260812 06:36:16.196172  7758 mvcc.cc:204] Tried to move back new op lower bound from 7317571895987036160 to 7317571894415798272. Current Snapshot: MvccSnapshot[applied={T|T < 7317571895987036160 or (T in {7317571895987036160})}]
W20260812 06:36:16.403921  6046 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.GetConsensusState from 127.0.0.1:53168 (request call id 24) took 129 ms (client timeout 100 ms). Trace:
W20260812 06:36:16.404044  6046 rpcz_store.cc:269] 0812 06:36:16.274180 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 06:36:16.275282 (+  1102us) service_pool.cc:224] Handling call
0812 06:36:16.403780 (+128498us) spinlock_profiling.cc:243] Waited 126 ms on lock 0x558235a22314. stack: 00007f90242e6d35 00007f90242e7325 00007f9023df382d 0000558232269b18 0000558232269b6b 00007f9027107df1 00007f9027107d5f 00007f9026345be1 00007f902742f5f1 00007f9025a90ac3 00007f9025a96e1a 00007f9025315f8f 00007f9025315843 00007f9025318541 00007f9025316e9e 00007f90253188cb
0812 06:36:16.403873 (+    93us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":289502464}
I20260812 06:36:18.052537  7790 raft_consensus.cc:493] T 7065bb403e3047dd9a0853c49318ef1f P ec6c8d818e744cdbb6b0d15b321010e0 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 222b20de5a1145c6b8a11df8374e8790)
I20260812 06:36:18.052714  7790 raft_consensus.cc:515] T 7065bb403e3047dd9a0853c49318ef1f P ec6c8d818e744cdbb6b0d15b321010e0 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df4308c852240859ada53d216ab69ae" member_type: VOTER last_known_addr { host: "127.4.177.6" port: 34269 } } peers { permanent_uuid: "42895a9bcba64aa484f6509605035d02" member_type: VOTER last_known_addr { host: "127.4.177.4" port: 38181 } } peers { permanent_uuid: "222b20de5a1145c6b8a11df8374e8790" member_type: VOTER last_known_addr { host: "127.4.177.5" port: 43521 } } peers { permanent_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 40897 } } peers { permanent_uuid: "8d90d61da62549208fe117b4384a469b" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 41539 } } peers { permanent_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" member_type: VOTER last_known_addr { host: "127.4.177.7" port: 46299 } } peers { permanent_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 41057 } }
I20260812 06:36:18.053419  7790 leader_election.cc:290] T 7065bb403e3047dd9a0853c49318ef1f P ec6c8d818e744cdbb6b0d15b321010e0 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 2df4308c852240859ada53d216ab69ae (127.4.177.6:34269), 42895a9bcba64aa484f6509605035d02 (127.4.177.4:38181), 222b20de5a1145c6b8a11df8374e8790 (127.4.177.5:43521), 7962f587ccd842ad8a3f6ed9a5d7707e (127.4.177.3:40897), 8d90d61da62549208fe117b4384a469b (127.4.177.2:41539), b037d17b21f8422fa5e41b70eb4dcd08 (127.4.177.7:46299)
I20260812 06:36:18.053835  7108 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" candidate_term: 3 candidate_status { last_received { term: 2 index: 441 } } ignore_live_leader: false dest_uuid: "2df4308c852240859ada53d216ab69ae" is_pre_election: true
I20260812 06:36:18.054015  7321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" candidate_term: 3 candidate_status { last_received { term: 2 index: 441 } } ignore_live_leader: false dest_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" is_pre_election: true
I20260812 06:36:18.054154  6479 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" candidate_term: 3 candidate_status { last_received { term: 2 index: 441 } } ignore_live_leader: false dest_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" is_pre_election: true
I20260812 06:36:18.054332  6703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" candidate_term: 3 candidate_status { last_received { term: 2 index: 441 } } ignore_live_leader: false dest_uuid: "42895a9bcba64aa484f6509605035d02" is_pre_election: true
I20260812 06:36:18.054804  5945 leader_election.cc:304] T 7065bb403e3047dd9a0853c49318ef1f P ec6c8d818e744cdbb6b0d15b321010e0 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 5 responses out of 7 voters: 1 yes votes; 4 no votes. yes voters: ec6c8d818e744cdbb6b0d15b321010e0; no voters: 2df4308c852240859ada53d216ab69ae, 42895a9bcba64aa484f6509605035d02, 7962f587ccd842ad8a3f6ed9a5d7707e, b037d17b21f8422fa5e41b70eb4dcd08
I20260812 06:36:18.055032  7790 raft_consensus.cc:2749] T 7065bb403e3047dd9a0853c49318ef1f P ec6c8d818e744cdbb6b0d15b321010e0 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260812 06:36:18.055016  6247 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" candidate_term: 3 candidate_status { last_received { term: 2 index: 441 } } ignore_live_leader: false dest_uuid: "8d90d61da62549208fe117b4384a469b" is_pre_election: true
I20260812 06:36:18.266461  7792 raft_consensus.cc:493] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 222b20de5a1145c6b8a11df8374e8790)
I20260812 06:36:18.266599  7792 raft_consensus.cc:515] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df4308c852240859ada53d216ab69ae" member_type: VOTER last_known_addr { host: "127.4.177.6" port: 34269 } } peers { permanent_uuid: "42895a9bcba64aa484f6509605035d02" member_type: VOTER last_known_addr { host: "127.4.177.4" port: 38181 } } peers { permanent_uuid: "222b20de5a1145c6b8a11df8374e8790" member_type: VOTER last_known_addr { host: "127.4.177.5" port: 43521 } } peers { permanent_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 40897 } } peers { permanent_uuid: "8d90d61da62549208fe117b4384a469b" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 41539 } } peers { permanent_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" member_type: VOTER last_known_addr { host: "127.4.177.7" port: 46299 } } peers { permanent_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 41057 } }
I20260812 06:36:18.267248  7792 leader_election.cc:290] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 2df4308c852240859ada53d216ab69ae (127.4.177.6:34269), 42895a9bcba64aa484f6509605035d02 (127.4.177.4:38181), 222b20de5a1145c6b8a11df8374e8790 (127.4.177.5:43521), 7962f587ccd842ad8a3f6ed9a5d7707e (127.4.177.3:40897), b037d17b21f8422fa5e41b70eb4dcd08 (127.4.177.7:46299), ec6c8d818e744cdbb6b0d15b321010e0 (127.4.177.1:41057)
I20260812 06:36:18.267604  7108 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "8d90d61da62549208fe117b4384a469b" candidate_term: 3 candidate_status { last_received { term: 2 index: 513 } } ignore_live_leader: false dest_uuid: "2df4308c852240859ada53d216ab69ae" is_pre_election: true
I20260812 06:36:18.267678  7321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "8d90d61da62549208fe117b4384a469b" candidate_term: 3 candidate_status { last_received { term: 2 index: 513 } } ignore_live_leader: false dest_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" is_pre_election: true
I20260812 06:36:18.267867  6047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "8d90d61da62549208fe117b4384a469b" candidate_term: 3 candidate_status { last_received { term: 2 index: 513 } } ignore_live_leader: false dest_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" is_pre_election: true
I20260812 06:36:18.267977  6703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "8d90d61da62549208fe117b4384a469b" candidate_term: 3 candidate_status { last_received { term: 2 index: 513 } } ignore_live_leader: false dest_uuid: "42895a9bcba64aa484f6509605035d02" is_pre_election: true
I20260812 06:36:18.268028  6047 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P ec6c8d818e744cdbb6b0d15b321010e0 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8d90d61da62549208fe117b4384a469b in term 2.
I20260812 06:36:18.268299  6479 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "8d90d61da62549208fe117b4384a469b" candidate_term: 3 candidate_status { last_received { term: 2 index: 513 } } ignore_live_leader: false dest_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" is_pre_election: true
I20260812 06:36:18.268723  6150 leader_election.cc:304] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 6 responses out of 7 voters: 2 yes votes; 4 no votes. yes voters: 8d90d61da62549208fe117b4384a469b, ec6c8d818e744cdbb6b0d15b321010e0; no voters: 2df4308c852240859ada53d216ab69ae, 42895a9bcba64aa484f6509605035d02, 7962f587ccd842ad8a3f6ed9a5d7707e, b037d17b21f8422fa5e41b70eb4dcd08
I20260812 06:36:18.268895  7792 raft_consensus.cc:2749] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260812 06:36:18.392338  7794 raft_consensus.cc:493] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 222b20de5a1145c6b8a11df8374e8790)
I20260812 06:36:18.392537  7794 raft_consensus.cc:515] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df4308c852240859ada53d216ab69ae" member_type: VOTER last_known_addr { host: "127.4.177.6" port: 34269 } } peers { permanent_uuid: "42895a9bcba64aa484f6509605035d02" member_type: VOTER last_known_addr { host: "127.4.177.4" port: 38181 } } peers { permanent_uuid: "222b20de5a1145c6b8a11df8374e8790" member_type: VOTER last_known_addr { host: "127.4.177.5" port: 43521 } } peers { permanent_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 40897 } } peers { permanent_uuid: "8d90d61da62549208fe117b4384a469b" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 41539 } } peers { permanent_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" member_type: VOTER last_known_addr { host: "127.4.177.7" port: 46299 } } peers { permanent_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 41057 } }
I20260812 06:36:18.393352  7794 leader_election.cc:290] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 42895a9bcba64aa484f6509605035d02 (127.4.177.4:38181), 222b20de5a1145c6b8a11df8374e8790 (127.4.177.5:43521), 7962f587ccd842ad8a3f6ed9a5d7707e (127.4.177.3:40897), 8d90d61da62549208fe117b4384a469b (127.4.177.2:41539), b037d17b21f8422fa5e41b70eb4dcd08 (127.4.177.7:46299), ec6c8d818e744cdbb6b0d15b321010e0 (127.4.177.1:41057)
I20260812 06:36:18.393525  7796 raft_consensus.cc:493] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 222b20de5a1145c6b8a11df8374e8790)
I20260812 06:36:18.393723  7796 raft_consensus.cc:515] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df4308c852240859ada53d216ab69ae" member_type: VOTER last_known_addr { host: "127.4.177.6" port: 34269 } } peers { permanent_uuid: "42895a9bcba64aa484f6509605035d02" member_type: VOTER last_known_addr { host: "127.4.177.4" port: 38181 } } peers { permanent_uuid: "222b20de5a1145c6b8a11df8374e8790" member_type: VOTER last_known_addr { host: "127.4.177.5" port: 43521 } } peers { permanent_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 40897 } } peers { permanent_uuid: "8d90d61da62549208fe117b4384a469b" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 41539 } } peers { permanent_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" member_type: VOTER last_known_addr { host: "127.4.177.7" port: 46299 } } peers { permanent_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 41057 } }
I20260812 06:36:18.393942  6703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "2df4308c852240859ada53d216ab69ae" candidate_term: 3 candidate_status { last_received { term: 2 index: 567 } } ignore_live_leader: false dest_uuid: "42895a9bcba64aa484f6509605035d02" is_pre_election: true
I20260812 06:36:18.393985  6047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "2df4308c852240859ada53d216ab69ae" candidate_term: 3 candidate_status { last_received { term: 2 index: 567 } } ignore_live_leader: false dest_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" is_pre_election: true
I20260812 06:36:18.394075  6703 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2df4308c852240859ada53d216ab69ae in term 2.
I20260812 06:36:18.394028  7321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "2df4308c852240859ada53d216ab69ae" candidate_term: 3 candidate_status { last_received { term: 2 index: 567 } } ignore_live_leader: false dest_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" is_pre_election: true
I20260812 06:36:18.394145  6047 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P ec6c8d818e744cdbb6b0d15b321010e0 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2df4308c852240859ada53d216ab69ae in term 2.
I20260812 06:36:18.394400  6479 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "2df4308c852240859ada53d216ab69ae" candidate_term: 3 candidate_status { last_received { term: 2 index: 567 } } ignore_live_leader: false dest_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" is_pre_election: true
I20260812 06:36:18.394399  6247 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "2df4308c852240859ada53d216ab69ae" candidate_term: 3 candidate_status { last_received { term: 2 index: 567 } } ignore_live_leader: false dest_uuid: "8d90d61da62549208fe117b4384a469b" is_pre_election: true
I20260812 06:36:18.394536  6247 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2df4308c852240859ada53d216ab69ae in term 2.
I20260812 06:36:18.394920  7321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "42895a9bcba64aa484f6509605035d02" candidate_term: 3 candidate_status { last_received { term: 2 index: 567 } } ignore_live_leader: false dest_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" is_pre_election: true
I20260812 06:36:18.395004  7796 leader_election.cc:290] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 2df4308c852240859ada53d216ab69ae (127.4.177.6:34269), 222b20de5a1145c6b8a11df8374e8790 (127.4.177.5:43521), 7962f587ccd842ad8a3f6ed9a5d7707e (127.4.177.3:40897), 8d90d61da62549208fe117b4384a469b (127.4.177.2:41539), b037d17b21f8422fa5e41b70eb4dcd08 (127.4.177.7:46299), ec6c8d818e744cdbb6b0d15b321010e0 (127.4.177.1:41057)
I20260812 06:36:18.395116  7108 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "42895a9bcba64aa484f6509605035d02" candidate_term: 3 candidate_status { last_received { term: 2 index: 567 } } ignore_live_leader: false dest_uuid: "2df4308c852240859ada53d216ab69ae" is_pre_election: true
I20260812 06:36:18.395258  7108 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 42895a9bcba64aa484f6509605035d02 in term 2.
I20260812 06:36:18.395284  6247 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "42895a9bcba64aa484f6509605035d02" candidate_term: 3 candidate_status { last_received { term: 2 index: 567 } } ignore_live_leader: false dest_uuid: "8d90d61da62549208fe117b4384a469b" is_pre_election: true
I20260812 06:36:18.395403  6247 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 42895a9bcba64aa484f6509605035d02 in term 2.
I20260812 06:36:18.395735  6479 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "42895a9bcba64aa484f6509605035d02" candidate_term: 3 candidate_status { last_received { term: 2 index: 567 } } ignore_live_leader: false dest_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" is_pre_election: true
I20260812 06:36:18.395838  6047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "42895a9bcba64aa484f6509605035d02" candidate_term: 3 candidate_status { last_received { term: 2 index: 567 } } ignore_live_leader: false dest_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" is_pre_election: true
I20260812 06:36:18.395942  7020 leader_election.cc:304] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 6 responses out of 7 voters: 4 yes votes; 2 no votes. yes voters: 2df4308c852240859ada53d216ab69ae, 42895a9bcba64aa484f6509605035d02, 8d90d61da62549208fe117b4384a469b, ec6c8d818e744cdbb6b0d15b321010e0; no voters: 7962f587ccd842ad8a3f6ed9a5d7707e, b037d17b21f8422fa5e41b70eb4dcd08
I20260812 06:36:18.395995  6047 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P ec6c8d818e744cdbb6b0d15b321010e0 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 42895a9bcba64aa484f6509605035d02 in term 2.
I20260812 06:36:18.396165  7794 raft_consensus.cc:2804] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260812 06:36:18.396261  7794 raft_consensus.cc:493] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [term 2 FOLLOWER]: Starting leader election (detected failure of leader 222b20de5a1145c6b8a11df8374e8790)
I20260812 06:36:18.396307  7794 raft_consensus.cc:3060] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [term 2 FOLLOWER]: Advancing to term 3
I20260812 06:36:18.396303  6597 leader_election.cc:304] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 6 responses out of 7 voters: 4 yes votes; 2 no votes. yes voters: 2df4308c852240859ada53d216ab69ae, 42895a9bcba64aa484f6509605035d02, 8d90d61da62549208fe117b4384a469b, ec6c8d818e744cdbb6b0d15b321010e0; no voters: 7962f587ccd842ad8a3f6ed9a5d7707e, b037d17b21f8422fa5e41b70eb4dcd08
I20260812 06:36:18.396486  7796 raft_consensus.cc:2804] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260812 06:36:18.396548  7796 raft_consensus.cc:493] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 222b20de5a1145c6b8a11df8374e8790)
I20260812 06:36:18.396579  7796 raft_consensus.cc:3060] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [term 2 FOLLOWER]: Advancing to term 3
I20260812 06:36:18.397320  7794 raft_consensus.cc:515] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df4308c852240859ada53d216ab69ae" member_type: VOTER last_known_addr { host: "127.4.177.6" port: 34269 } } peers { permanent_uuid: "42895a9bcba64aa484f6509605035d02" member_type: VOTER last_known_addr { host: "127.4.177.4" port: 38181 } } peers { permanent_uuid: "222b20de5a1145c6b8a11df8374e8790" member_type: VOTER last_known_addr { host: "127.4.177.5" port: 43521 } } peers { permanent_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 40897 } } peers { permanent_uuid: "8d90d61da62549208fe117b4384a469b" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 41539 } } peers { permanent_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" member_type: VOTER last_known_addr { host: "127.4.177.7" port: 46299 } } peers { permanent_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 41057 } }
I20260812 06:36:18.397533  7796 raft_consensus.cc:515] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df4308c852240859ada53d216ab69ae" member_type: VOTER last_known_addr { host: "127.4.177.6" port: 34269 } } peers { permanent_uuid: "42895a9bcba64aa484f6509605035d02" member_type: VOTER last_known_addr { host: "127.4.177.4" port: 38181 } } peers { permanent_uuid: "222b20de5a1145c6b8a11df8374e8790" member_type: VOTER last_known_addr { host: "127.4.177.5" port: 43521 } } peers { permanent_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 40897 } } peers { permanent_uuid: "8d90d61da62549208fe117b4384a469b" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 41539 } } peers { permanent_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" member_type: VOTER last_known_addr { host: "127.4.177.7" port: 46299 } } peers { permanent_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 41057 } }
I20260812 06:36:18.398116  7794 leader_election.cc:290] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [CANDIDATE]: Term 3 election: Requested vote from peers 42895a9bcba64aa484f6509605035d02 (127.4.177.4:38181), 222b20de5a1145c6b8a11df8374e8790 (127.4.177.5:43521), 7962f587ccd842ad8a3f6ed9a5d7707e (127.4.177.3:40897), 8d90d61da62549208fe117b4384a469b (127.4.177.2:41539), b037d17b21f8422fa5e41b70eb4dcd08 (127.4.177.7:46299), ec6c8d818e744cdbb6b0d15b321010e0 (127.4.177.1:41057)
I20260812 06:36:18.398586  7321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "42895a9bcba64aa484f6509605035d02" candidate_term: 3 candidate_status { last_received { term: 2 index: 567 } } ignore_live_leader: false dest_uuid: "b037d17b21f8422fa5e41b70eb4dcd08"
I20260812 06:36:18.398623  6480 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "42895a9bcba64aa484f6509605035d02" candidate_term: 3 candidate_status { last_received { term: 2 index: 567 } } ignore_live_leader: false dest_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e"
I20260812 06:36:18.398684  6479 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "2df4308c852240859ada53d216ab69ae" candidate_term: 3 candidate_status { last_received { term: 2 index: 567 } } ignore_live_leader: false dest_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e"
I20260812 06:36:18.398761  6047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "2df4308c852240859ada53d216ab69ae" candidate_term: 3 candidate_status { last_received { term: 2 index: 567 } } ignore_live_leader: false dest_uuid: "ec6c8d818e744cdbb6b0d15b321010e0"
I20260812 06:36:18.398865  6047 raft_consensus.cc:3060] T 7065bb403e3047dd9a0853c49318ef1f P ec6c8d818e744cdbb6b0d15b321010e0 [term 2 FOLLOWER]: Advancing to term 3
I20260812 06:36:18.399086  6247 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "2df4308c852240859ada53d216ab69ae" candidate_term: 3 candidate_status { last_received { term: 2 index: 567 } } ignore_live_leader: false dest_uuid: "8d90d61da62549208fe117b4384a469b"
I20260812 06:36:18.399195  7796 leader_election.cc:290] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [CANDIDATE]: Term 3 election: Requested vote from peers 2df4308c852240859ada53d216ab69ae (127.4.177.6:34269), 222b20de5a1145c6b8a11df8374e8790 (127.4.177.5:43521), 7962f587ccd842ad8a3f6ed9a5d7707e (127.4.177.3:40897), 8d90d61da62549208fe117b4384a469b (127.4.177.2:41539), b037d17b21f8422fa5e41b70eb4dcd08 (127.4.177.7:46299), ec6c8d818e744cdbb6b0d15b321010e0 (127.4.177.1:41057)
I20260812 06:36:18.399202  6249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "42895a9bcba64aa484f6509605035d02" candidate_term: 3 candidate_status { last_received { term: 2 index: 567 } } ignore_live_leader: false dest_uuid: "8d90d61da62549208fe117b4384a469b"
I20260812 06:36:18.399317  6703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "2df4308c852240859ada53d216ab69ae" candidate_term: 3 candidate_status { last_received { term: 2 index: 567 } } ignore_live_leader: false dest_uuid: "42895a9bcba64aa484f6509605035d02"
I20260812 06:36:18.399489  6703 raft_consensus.cc:2393] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 2df4308c852240859ada53d216ab69ae in current term 3: Already voted for candidate 42895a9bcba64aa484f6509605035d02 in this term.
I20260812 06:36:18.399829  7321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "2df4308c852240859ada53d216ab69ae" candidate_term: 3 candidate_status { last_received { term: 2 index: 567 } } ignore_live_leader: false dest_uuid: "b037d17b21f8422fa5e41b70eb4dcd08"
I20260812 06:36:18.399837  7108 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "42895a9bcba64aa484f6509605035d02" candidate_term: 3 candidate_status { last_received { term: 2 index: 567 } } ignore_live_leader: false dest_uuid: "2df4308c852240859ada53d216ab69ae"
I20260812 06:36:18.399204  6247 raft_consensus.cc:3060] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b [term 2 FOLLOWER]: Advancing to term 3
I20260812 06:36:18.399996  7108 raft_consensus.cc:2393] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 42895a9bcba64aa484f6509605035d02 in current term 3: Already voted for candidate 2df4308c852240859ada53d216ab69ae in this term.
I20260812 06:36:18.400127  6047 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P ec6c8d818e744cdbb6b0d15b321010e0 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2df4308c852240859ada53d216ab69ae in term 3.
I20260812 06:36:18.401002  6247 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2df4308c852240859ada53d216ab69ae in term 3.
I20260812 06:36:18.401441  6597 leader_election.cc:304] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 5 responses out of 7 voters: 1 yes votes; 4 no votes. yes voters: 42895a9bcba64aa484f6509605035d02; no voters: 2df4308c852240859ada53d216ab69ae, 7962f587ccd842ad8a3f6ed9a5d7707e, 8d90d61da62549208fe117b4384a469b, b037d17b21f8422fa5e41b70eb4dcd08
I20260812 06:36:18.401592  6047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "42895a9bcba64aa484f6509605035d02" candidate_term: 3 candidate_status { last_received { term: 2 index: 567 } } ignore_live_leader: false dest_uuid: "ec6c8d818e744cdbb6b0d15b321010e0"
I20260812 06:36:18.401651  7796 raft_consensus.cc:2749] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260812 06:36:18.401760  6047 raft_consensus.cc:2393] T 7065bb403e3047dd9a0853c49318ef1f P ec6c8d818e744cdbb6b0d15b321010e0 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 42895a9bcba64aa484f6509605035d02 in current term 3: Already voted for candidate 2df4308c852240859ada53d216ab69ae in this term.
I20260812 06:36:18.725549  7797 raft_consensus.cc:493] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 222b20de5a1145c6b8a11df8374e8790)
I20260812 06:36:18.725688  7797 raft_consensus.cc:515] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df4308c852240859ada53d216ab69ae" member_type: VOTER last_known_addr { host: "127.4.177.6" port: 34269 } } peers { permanent_uuid: "42895a9bcba64aa484f6509605035d02" member_type: VOTER last_known_addr { host: "127.4.177.4" port: 38181 } } peers { permanent_uuid: "222b20de5a1145c6b8a11df8374e8790" member_type: VOTER last_known_addr { host: "127.4.177.5" port: 43521 } } peers { permanent_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 40897 } } peers { permanent_uuid: "8d90d61da62549208fe117b4384a469b" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 41539 } } peers { permanent_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" member_type: VOTER last_known_addr { host: "127.4.177.7" port: 46299 } } peers { permanent_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 41057 } }
I20260812 06:36:18.726390  7797 leader_election.cc:290] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 2df4308c852240859ada53d216ab69ae (127.4.177.6:34269), 42895a9bcba64aa484f6509605035d02 (127.4.177.4:38181), 222b20de5a1145c6b8a11df8374e8790 (127.4.177.5:43521), 7962f587ccd842ad8a3f6ed9a5d7707e (127.4.177.3:40897), 8d90d61da62549208fe117b4384a469b (127.4.177.2:41539), ec6c8d818e744cdbb6b0d15b321010e0 (127.4.177.1:41057)
I20260812 06:36:18.726866  7108 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" candidate_term: 3 candidate_status { last_received { term: 2 index: 473 } } ignore_live_leader: false dest_uuid: "2df4308c852240859ada53d216ab69ae" is_pre_election: true
I20260812 06:36:18.726972  6479 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" candidate_term: 3 candidate_status { last_received { term: 2 index: 473 } } ignore_live_leader: false dest_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" is_pre_election: true
I20260812 06:36:18.727063  7108 raft_consensus.cc:2393] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b037d17b21f8422fa5e41b70eb4dcd08 in current term 3: Already voted for candidate 2df4308c852240859ada53d216ab69ae in this term.
I20260812 06:36:18.726989  6703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" candidate_term: 3 candidate_status { last_received { term: 2 index: 473 } } ignore_live_leader: false dest_uuid: "42895a9bcba64aa484f6509605035d02" is_pre_election: true
I20260812 06:36:18.727190  6703 raft_consensus.cc:2393] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b037d17b21f8422fa5e41b70eb4dcd08 in current term 3: Already voted for candidate 42895a9bcba64aa484f6509605035d02 in this term.
I20260812 06:36:18.727270  6247 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" candidate_term: 3 candidate_status { last_received { term: 2 index: 473 } } ignore_live_leader: false dest_uuid: "8d90d61da62549208fe117b4384a469b" is_pre_election: true
I20260812 06:36:18.727473  6247 raft_consensus.cc:2393] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b037d17b21f8422fa5e41b70eb4dcd08 in current term 3: Already voted for candidate 2df4308c852240859ada53d216ab69ae in this term.
I20260812 06:36:18.727490  6047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" candidate_term: 3 candidate_status { last_received { term: 2 index: 473 } } ignore_live_leader: false dest_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" is_pre_election: true
I20260812 06:36:18.727743  6047 raft_consensus.cc:2393] T 7065bb403e3047dd9a0853c49318ef1f P ec6c8d818e744cdbb6b0d15b321010e0 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b037d17b21f8422fa5e41b70eb4dcd08 in current term 3: Already voted for candidate 2df4308c852240859ada53d216ab69ae in this term.
I20260812 06:36:18.727888  7211 leader_election.cc:304] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 5 responses out of 7 voters: 1 yes votes; 4 no votes. yes voters: b037d17b21f8422fa5e41b70eb4dcd08; no voters: 2df4308c852240859ada53d216ab69ae, 42895a9bcba64aa484f6509605035d02, 7962f587ccd842ad8a3f6ed9a5d7707e, 8d90d61da62549208fe117b4384a469b
I20260812 06:36:18.728084  7797 raft_consensus.cc:3060] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08 [term 2 FOLLOWER]: Advancing to term 3
I20260812 06:36:18.728463  7798 raft_consensus.cc:493] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 222b20de5a1145c6b8a11df8374e8790)
I20260812 06:36:18.728567  7798 raft_consensus.cc:515] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df4308c852240859ada53d216ab69ae" member_type: VOTER last_known_addr { host: "127.4.177.6" port: 34269 } } peers { permanent_uuid: "42895a9bcba64aa484f6509605035d02" member_type: VOTER last_known_addr { host: "127.4.177.4" port: 38181 } } peers { permanent_uuid: "222b20de5a1145c6b8a11df8374e8790" member_type: VOTER last_known_addr { host: "127.4.177.5" port: 43521 } } peers { permanent_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 40897 } } peers { permanent_uuid: "8d90d61da62549208fe117b4384a469b" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 41539 } } peers { permanent_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" member_type: VOTER last_known_addr { host: "127.4.177.7" port: 46299 } } peers { permanent_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 41057 } }
I20260812 06:36:18.729434  7108 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" candidate_term: 3 candidate_status { last_received { term: 2 index: 561 } } ignore_live_leader: false dest_uuid: "2df4308c852240859ada53d216ab69ae" is_pre_election: true
I20260812 06:36:18.729538  7321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" candidate_term: 3 candidate_status { last_received { term: 2 index: 561 } } ignore_live_leader: false dest_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" is_pre_election: true
I20260812 06:36:18.729590  7108 raft_consensus.cc:2393] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7962f587ccd842ad8a3f6ed9a5d7707e in current term 3: Already voted for candidate 2df4308c852240859ada53d216ab69ae in this term.
I20260812 06:36:18.729635  6703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" candidate_term: 3 candidate_status { last_received { term: 2 index: 561 } } ignore_live_leader: false dest_uuid: "42895a9bcba64aa484f6509605035d02" is_pre_election: true
I20260812 06:36:18.729672  6247 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" candidate_term: 3 candidate_status { last_received { term: 2 index: 561 } } ignore_live_leader: false dest_uuid: "8d90d61da62549208fe117b4384a469b" is_pre_election: true
I20260812 06:36:18.729770  6703 raft_consensus.cc:2393] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7962f587ccd842ad8a3f6ed9a5d7707e in current term 3: Already voted for candidate 42895a9bcba64aa484f6509605035d02 in this term.
I20260812 06:36:18.729807  6247 raft_consensus.cc:2393] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7962f587ccd842ad8a3f6ed9a5d7707e in current term 3: Already voted for candidate 2df4308c852240859ada53d216ab69ae in this term.
I20260812 06:36:18.730051  7797 raft_consensus.cc:2749] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260812 06:36:18.730154  7798 leader_election.cc:290] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 2df4308c852240859ada53d216ab69ae (127.4.177.6:34269), 42895a9bcba64aa484f6509605035d02 (127.4.177.4:38181), 222b20de5a1145c6b8a11df8374e8790 (127.4.177.5:43521), 8d90d61da62549208fe117b4384a469b (127.4.177.2:41539), b037d17b21f8422fa5e41b70eb4dcd08 (127.4.177.7:46299), ec6c8d818e744cdbb6b0d15b321010e0 (127.4.177.1:41057)
I20260812 06:36:18.730373  7321 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7962f587ccd842ad8a3f6ed9a5d7707e in term 3.
I20260812 06:36:18.730670  6047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" candidate_term: 3 candidate_status { last_received { term: 2 index: 561 } } ignore_live_leader: false dest_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" is_pre_election: true
I20260812 06:36:18.730834  6047 raft_consensus.cc:2393] T 7065bb403e3047dd9a0853c49318ef1f P ec6c8d818e744cdbb6b0d15b321010e0 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7962f587ccd842ad8a3f6ed9a5d7707e in current term 3: Already voted for candidate 2df4308c852240859ada53d216ab69ae in this term.
I20260812 06:36:18.731137  6369 leader_election.cc:304] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 6 responses out of 7 voters: 2 yes votes; 4 no votes. yes voters: 7962f587ccd842ad8a3f6ed9a5d7707e, b037d17b21f8422fa5e41b70eb4dcd08; no voters: 2df4308c852240859ada53d216ab69ae, 42895a9bcba64aa484f6509605035d02, 8d90d61da62549208fe117b4384a469b, ec6c8d818e744cdbb6b0d15b321010e0
I20260812 06:36:18.731310  7798 raft_consensus.cc:3060] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [term 2 FOLLOWER]: Advancing to term 3
I20260812 06:36:18.732219  7798 raft_consensus.cc:2749] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260812 06:36:19.572954  5945 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.4.177.5:43521 timed out after 1.518s (SENT)
W20260812 06:36:19.924431  7020 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.4.177.5:43521 timed out after 1.530s (SENT)
W20260812 06:36:20.023150  6150 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.4.177.5:43521 timed out after 1.754s (SENT)
W20260812 06:36:20.205273  6597 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.4.177.5:43521 timed out after 1.809s (SENT)
W20260812 06:36:20.395393  6369 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.4.177.5:43521 timed out after 1.664s (SENT)
I20260812 06:36:20.450917  7808 raft_consensus.cc:493] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 06:36:20.451084  7808 raft_consensus.cc:515] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df4308c852240859ada53d216ab69ae" member_type: VOTER last_known_addr { host: "127.4.177.6" port: 34269 } } peers { permanent_uuid: "42895a9bcba64aa484f6509605035d02" member_type: VOTER last_known_addr { host: "127.4.177.4" port: 38181 } } peers { permanent_uuid: "222b20de5a1145c6b8a11df8374e8790" member_type: VOTER last_known_addr { host: "127.4.177.5" port: 43521 } } peers { permanent_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 40897 } } peers { permanent_uuid: "8d90d61da62549208fe117b4384a469b" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 41539 } } peers { permanent_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" member_type: VOTER last_known_addr { host: "127.4.177.7" port: 46299 } } peers { permanent_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 41057 } }
I20260812 06:36:20.451727  7808 leader_election.cc:290] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 2df4308c852240859ada53d216ab69ae (127.4.177.6:34269), 42895a9bcba64aa484f6509605035d02 (127.4.177.4:38181), 222b20de5a1145c6b8a11df8374e8790 (127.4.177.5:43521), 7962f587ccd842ad8a3f6ed9a5d7707e (127.4.177.3:40897), 8d90d61da62549208fe117b4384a469b (127.4.177.2:41539), ec6c8d818e744cdbb6b0d15b321010e0 (127.4.177.1:41057)
I20260812 06:36:20.452118  7108 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" candidate_term: 4 candidate_status { last_received { term: 2 index: 473 } } ignore_live_leader: false dest_uuid: "2df4308c852240859ada53d216ab69ae" is_pre_election: true
I20260812 06:36:20.452149  6703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" candidate_term: 4 candidate_status { last_received { term: 2 index: 473 } } ignore_live_leader: false dest_uuid: "42895a9bcba64aa484f6509605035d02" is_pre_election: true
I20260812 06:36:20.452298  7108 raft_consensus.cc:2410] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b037d17b21f8422fa5e41b70eb4dcd08 for term 4 because replica has last-logged OpId of term: 2 index: 567, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 473.
I20260812 06:36:20.452308  6479 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" candidate_term: 4 candidate_status { last_received { term: 2 index: 473 } } ignore_live_leader: false dest_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" is_pre_election: true
I20260812 06:36:20.452358  6703 raft_consensus.cc:2410] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b037d17b21f8422fa5e41b70eb4dcd08 for term 4 because replica has last-logged OpId of term: 2 index: 567, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 473.
I20260812 06:36:20.452383  6047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" candidate_term: 4 candidate_status { last_received { term: 2 index: 473 } } ignore_live_leader: false dest_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" is_pre_election: true
I20260812 06:36:20.452456  6479 raft_consensus.cc:2410] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b037d17b21f8422fa5e41b70eb4dcd08 for term 4 because replica has last-logged OpId of term: 2 index: 561, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 473.
I20260812 06:36:20.452510  6047 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P ec6c8d818e744cdbb6b0d15b321010e0 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b037d17b21f8422fa5e41b70eb4dcd08 in term 3.
I20260812 06:36:20.452991  6247 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" candidate_term: 4 candidate_status { last_received { term: 2 index: 473 } } ignore_live_leader: false dest_uuid: "8d90d61da62549208fe117b4384a469b" is_pre_election: true
I20260812 06:36:20.453166  6247 raft_consensus.cc:2410] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b037d17b21f8422fa5e41b70eb4dcd08 for term 4 because replica has last-logged OpId of term: 2 index: 513, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 473.
I20260812 06:36:20.453534  7211 leader_election.cc:304] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 6 responses out of 7 voters: 2 yes votes; 4 no votes. yes voters: b037d17b21f8422fa5e41b70eb4dcd08, ec6c8d818e744cdbb6b0d15b321010e0; no voters: 2df4308c852240859ada53d216ab69ae, 42895a9bcba64aa484f6509605035d02, 7962f587ccd842ad8a3f6ed9a5d7707e, 8d90d61da62549208fe117b4384a469b
I20260812 06:36:20.453706  7808 raft_consensus.cc:2749] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260812 06:36:20.459055  7809 raft_consensus.cc:493] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 06:36:20.459199  7809 raft_consensus.cc:515] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df4308c852240859ada53d216ab69ae" member_type: VOTER last_known_addr { host: "127.4.177.6" port: 34269 } } peers { permanent_uuid: "42895a9bcba64aa484f6509605035d02" member_type: VOTER last_known_addr { host: "127.4.177.4" port: 38181 } } peers { permanent_uuid: "222b20de5a1145c6b8a11df8374e8790" member_type: VOTER last_known_addr { host: "127.4.177.5" port: 43521 } } peers { permanent_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 40897 } } peers { permanent_uuid: "8d90d61da62549208fe117b4384a469b" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 41539 } } peers { permanent_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" member_type: VOTER last_known_addr { host: "127.4.177.7" port: 46299 } } peers { permanent_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 41057 } }
I20260812 06:36:20.459813  7809 leader_election.cc:290] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 2df4308c852240859ada53d216ab69ae (127.4.177.6:34269), 42895a9bcba64aa484f6509605035d02 (127.4.177.4:38181), 222b20de5a1145c6b8a11df8374e8790 (127.4.177.5:43521), 8d90d61da62549208fe117b4384a469b (127.4.177.2:41539), b037d17b21f8422fa5e41b70eb4dcd08 (127.4.177.7:46299), ec6c8d818e744cdbb6b0d15b321010e0 (127.4.177.1:41057)
I20260812 06:36:20.460150  6703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" candidate_term: 4 candidate_status { last_received { term: 2 index: 561 } } ignore_live_leader: false dest_uuid: "42895a9bcba64aa484f6509605035d02" is_pre_election: true
I20260812 06:36:20.460189  7321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" candidate_term: 4 candidate_status { last_received { term: 2 index: 561 } } ignore_live_leader: false dest_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" is_pre_election: true
I20260812 06:36:20.460249  7108 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" candidate_term: 4 candidate_status { last_received { term: 2 index: 561 } } ignore_live_leader: false dest_uuid: "2df4308c852240859ada53d216ab69ae" is_pre_election: true
I20260812 06:36:20.460314  6703 raft_consensus.cc:2410] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7962f587ccd842ad8a3f6ed9a5d7707e for term 4 because replica has last-logged OpId of term: 2 index: 567, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 561.
I20260812 06:36:20.460328  7321 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7962f587ccd842ad8a3f6ed9a5d7707e in term 3.
I20260812 06:36:20.460419  7108 raft_consensus.cc:2410] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7962f587ccd842ad8a3f6ed9a5d7707e for term 4 because replica has last-logged OpId of term: 2 index: 567, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 561.
I20260812 06:36:20.460632  6247 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" candidate_term: 4 candidate_status { last_received { term: 2 index: 561 } } ignore_live_leader: false dest_uuid: "8d90d61da62549208fe117b4384a469b" is_pre_election: true
I20260812 06:36:20.460659  6047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" candidate_term: 4 candidate_status { last_received { term: 2 index: 561 } } ignore_live_leader: false dest_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" is_pre_election: true
I20260812 06:36:20.460760  6247 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7962f587ccd842ad8a3f6ed9a5d7707e in term 3.
I20260812 06:36:20.460777  6047 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P ec6c8d818e744cdbb6b0d15b321010e0 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7962f587ccd842ad8a3f6ed9a5d7707e in term 3.
I20260812 06:36:20.461084  6369 leader_election.cc:304] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 6 responses out of 7 voters: 4 yes votes; 2 no votes. yes voters: 7962f587ccd842ad8a3f6ed9a5d7707e, 8d90d61da62549208fe117b4384a469b, b037d17b21f8422fa5e41b70eb4dcd08, ec6c8d818e744cdbb6b0d15b321010e0; no voters: 2df4308c852240859ada53d216ab69ae, 42895a9bcba64aa484f6509605035d02
I20260812 06:36:20.461287  7809 raft_consensus.cc:2804] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260812 06:36:20.461344  7809 raft_consensus.cc:493] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 06:36:20.461421  7809 raft_consensus.cc:3060] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [term 3 FOLLOWER]: Advancing to term 4
I20260812 06:36:20.462440  7809 raft_consensus.cc:515] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df4308c852240859ada53d216ab69ae" member_type: VOTER last_known_addr { host: "127.4.177.6" port: 34269 } } peers { permanent_uuid: "42895a9bcba64aa484f6509605035d02" member_type: VOTER last_known_addr { host: "127.4.177.4" port: 38181 } } peers { permanent_uuid: "222b20de5a1145c6b8a11df8374e8790" member_type: VOTER last_known_addr { host: "127.4.177.5" port: 43521 } } peers { permanent_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 40897 } } peers { permanent_uuid: "8d90d61da62549208fe117b4384a469b" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 41539 } } peers { permanent_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" member_type: VOTER last_known_addr { host: "127.4.177.7" port: 46299 } } peers { permanent_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 41057 } }
I20260812 06:36:20.462992  7809 leader_election.cc:290] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [CANDIDATE]: Term 4 election: Requested vote from peers 2df4308c852240859ada53d216ab69ae (127.4.177.6:34269), 42895a9bcba64aa484f6509605035d02 (127.4.177.4:38181), 222b20de5a1145c6b8a11df8374e8790 (127.4.177.5:43521), 8d90d61da62549208fe117b4384a469b (127.4.177.2:41539), b037d17b21f8422fa5e41b70eb4dcd08 (127.4.177.7:46299), ec6c8d818e744cdbb6b0d15b321010e0 (127.4.177.1:41057)
I20260812 06:36:20.463284  7108 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" candidate_term: 4 candidate_status { last_received { term: 2 index: 561 } } ignore_live_leader: false dest_uuid: "2df4308c852240859ada53d216ab69ae"
I20260812 06:36:20.463284  6703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" candidate_term: 4 candidate_status { last_received { term: 2 index: 561 } } ignore_live_leader: false dest_uuid: "42895a9bcba64aa484f6509605035d02"
I20260812 06:36:20.463291  7321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" candidate_term: 4 candidate_status { last_received { term: 2 index: 561 } } ignore_live_leader: false dest_uuid: "b037d17b21f8422fa5e41b70eb4dcd08"
I20260812 06:36:20.463389  6703 raft_consensus.cc:3060] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [term 3 FOLLOWER]: Advancing to term 4
I20260812 06:36:20.463397  7321 raft_consensus.cc:3060] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08 [term 3 FOLLOWER]: Advancing to term 4
I20260812 06:36:20.463405  7108 raft_consensus.cc:3060] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [term 3 FOLLOWER]: Advancing to term 4
I20260812 06:36:20.463425  6247 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" candidate_term: 4 candidate_status { last_received { term: 2 index: 561 } } ignore_live_leader: false dest_uuid: "8d90d61da62549208fe117b4384a469b"
I20260812 06:36:20.463544  6247 raft_consensus.cc:3060] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b [term 3 FOLLOWER]: Advancing to term 4
I20260812 06:36:20.464407  6703 raft_consensus.cc:2410] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 7962f587ccd842ad8a3f6ed9a5d7707e for term 4 because replica has last-logged OpId of term: 2 index: 567, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 561.
I20260812 06:36:20.464409  7108 raft_consensus.cc:2410] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 7962f587ccd842ad8a3f6ed9a5d7707e for term 4 because replica has last-logged OpId of term: 2 index: 567, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 561.
I20260812 06:36:20.464454  7321 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7962f587ccd842ad8a3f6ed9a5d7707e in term 4.
I20260812 06:36:20.464494  6247 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7962f587ccd842ad8a3f6ed9a5d7707e in term 4.
I20260812 06:36:20.464854  6047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" candidate_term: 4 candidate_status { last_received { term: 2 index: 561 } } ignore_live_leader: false dest_uuid: "ec6c8d818e744cdbb6b0d15b321010e0"
I20260812 06:36:20.464988  6047 raft_consensus.cc:3060] T 7065bb403e3047dd9a0853c49318ef1f P ec6c8d818e744cdbb6b0d15b321010e0 [term 3 FOLLOWER]: Advancing to term 4
I20260812 06:36:20.465780  6047 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P ec6c8d818e744cdbb6b0d15b321010e0 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7962f587ccd842ad8a3f6ed9a5d7707e in term 4.
I20260812 06:36:20.466079  6369 leader_election.cc:304] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 6 responses out of 7 voters: 4 yes votes; 2 no votes. yes voters: 7962f587ccd842ad8a3f6ed9a5d7707e, 8d90d61da62549208fe117b4384a469b, b037d17b21f8422fa5e41b70eb4dcd08, ec6c8d818e744cdbb6b0d15b321010e0; no voters: 2df4308c852240859ada53d216ab69ae, 42895a9bcba64aa484f6509605035d02
I20260812 06:36:20.466295  7809 raft_consensus.cc:2804] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [term 4 FOLLOWER]: Leader election won for term 4
I20260812 06:36:20.468475  7809 raft_consensus.cc:697] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [term 4 LEADER]: Becoming Leader. State: Replica: 7962f587ccd842ad8a3f6ed9a5d7707e, State: Running, Role: LEADER
I20260812 06:36:20.468693  7809 consensus_queue.cc:237] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 559, Committed index: 559, Last appended: 2.561, Last appended by leader: 561, Current term: 4, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df4308c852240859ada53d216ab69ae" member_type: VOTER last_known_addr { host: "127.4.177.6" port: 34269 } } peers { permanent_uuid: "42895a9bcba64aa484f6509605035d02" member_type: VOTER last_known_addr { host: "127.4.177.4" port: 38181 } } peers { permanent_uuid: "222b20de5a1145c6b8a11df8374e8790" member_type: VOTER last_known_addr { host: "127.4.177.5" port: 43521 } } peers { permanent_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 40897 } } peers { permanent_uuid: "8d90d61da62549208fe117b4384a469b" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 41539 } } peers { permanent_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" member_type: VOTER last_known_addr { host: "127.4.177.7" port: 46299 } } peers { permanent_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 41057 } }
I20260812 06:36:20.470784  5812 catalog_manager.cc:5719] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e reported cstate change: term changed from 2 to 4, leader changed from 222b20de5a1145c6b8a11df8374e8790 (127.4.177.5) to 7962f587ccd842ad8a3f6ed9a5d7707e (127.4.177.3). New cstate: current_term: 4 leader_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df4308c852240859ada53d216ab69ae" member_type: VOTER last_known_addr { host: "127.4.177.6" port: 34269 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "42895a9bcba64aa484f6509605035d02" member_type: VOTER last_known_addr { host: "127.4.177.4" port: 38181 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "222b20de5a1145c6b8a11df8374e8790" member_type: VOTER last_known_addr { host: "127.4.177.5" port: 43521 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 40897 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8d90d61da62549208fe117b4384a469b" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 41539 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" member_type: VOTER last_known_addr { host: "127.4.177.7" port: 46299 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 41057 } health_report { overall_health: UNKNOWN } } }
W20260812 06:36:20.592598  6818 connection.cc:256] RPC call timeout handler was delayed by -3.0364170069806278s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1s for a response
W20260812 06:36:20.592911  6821 connection.cc:256] RPC call timeout handler was delayed by -3.1436535890679806s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1s for a response
W20260812 06:36:20.592993  6821 connection.cc:256] RPC call timeout handler was delayed by -2.928567216033116s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1s for a response
W20260812 06:36:20.593035  6821 connection.cc:256] RPC call timeout handler was delayed by -2.8140281070955098s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1s for a response
I20260812 06:36:20.593794  6917 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" candidate_term: 4 candidate_status { last_received { term: 2 index: 561 } } ignore_live_leader: false dest_uuid: "222b20de5a1145c6b8a11df8374e8790"
I20260812 06:36:20.594806  6909 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "42895a9bcba64aa484f6509605035d02" candidate_term: 3 candidate_status { last_received { term: 2 index: 567 } } ignore_live_leader: false dest_uuid: "222b20de5a1145c6b8a11df8374e8790"
I20260812 06:36:20.594898  6894 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" candidate_term: 3 candidate_status { last_received { term: 2 index: 441 } } ignore_live_leader: false dest_uuid: "222b20de5a1145c6b8a11df8374e8790" is_pre_election: true
I20260812 06:36:20.594945  6917 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" candidate_term: 3 candidate_status { last_received { term: 2 index: 441 } } ignore_live_leader: false dest_uuid: "222b20de5a1145c6b8a11df8374e8790" is_pre_election: true
I20260812 06:36:20.595112  6911 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "42895a9bcba64aa484f6509605035d02" candidate_term: 3 candidate_status { last_received { term: 2 index: 567 } } ignore_live_leader: false dest_uuid: "222b20de5a1145c6b8a11df8374e8790" is_pre_election: true
I20260812 06:36:20.595213  6916 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" candidate_term: 3 candidate_status { last_received { term: 2 index: 473 } } ignore_live_leader: false dest_uuid: "222b20de5a1145c6b8a11df8374e8790" is_pre_election: true
I20260812 06:36:20.595274  6915 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" candidate_term: 4 candidate_status { last_received { term: 2 index: 473 } } ignore_live_leader: false dest_uuid: "222b20de5a1145c6b8a11df8374e8790" is_pre_election: true
I20260812 06:36:20.595114  6895 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "8d90d61da62549208fe117b4384a469b" candidate_term: 3 candidate_status { last_received { term: 2 index: 513 } } ignore_live_leader: false dest_uuid: "222b20de5a1145c6b8a11df8374e8790" is_pre_election: true
I20260812 06:36:20.595425  6897 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "8d90d61da62549208fe117b4384a469b" candidate_term: 3 candidate_status { last_received { term: 2 index: 513 } } ignore_live_leader: false dest_uuid: "222b20de5a1145c6b8a11df8374e8790" is_pre_election: true
I20260812 06:36:20.595499  6918 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" candidate_term: 4 candidate_status { last_received { term: 2 index: 561 } } ignore_live_leader: false dest_uuid: "222b20de5a1145c6b8a11df8374e8790" is_pre_election: true
I20260812 06:36:20.595608  6914 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "2df4308c852240859ada53d216ab69ae" candidate_term: 3 candidate_status { last_received { term: 2 index: 567 } } ignore_live_leader: false dest_uuid: "222b20de5a1145c6b8a11df8374e8790" is_pre_election: true
I20260812 06:36:20.595503  6912 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "2df4308c852240859ada53d216ab69ae" candidate_term: 3 candidate_status { last_received { term: 2 index: 567 } } ignore_live_leader: false dest_uuid: "222b20de5a1145c6b8a11df8374e8790"
I20260812 06:36:20.595817  6910 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "2df4308c852240859ada53d216ab69ae" candidate_term: 3 candidate_status { last_received { term: 2 index: 567 } } ignore_live_leader: false dest_uuid: "222b20de5a1145c6b8a11df8374e8790" is_pre_election: true
I20260812 06:36:20.595825  6899 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "42895a9bcba64aa484f6509605035d02" candidate_term: 3 candidate_status { last_received { term: 2 index: 567 } } ignore_live_leader: false dest_uuid: "222b20de5a1145c6b8a11df8374e8790"
I20260812 06:36:20.596184  6907 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "42895a9bcba64aa484f6509605035d02" candidate_term: 3 candidate_status { last_received { term: 2 index: 567 } } ignore_live_leader: false dest_uuid: "222b20de5a1145c6b8a11df8374e8790" is_pre_election: true
I20260812 06:36:20.596397  6703 raft_consensus.cc:1240] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [term 4 FOLLOWER]: Rejecting Update request from peer 222b20de5a1145c6b8a11df8374e8790 for earlier term 2. Current term is 4. Ops: []
I20260812 06:36:20.596493  6908 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "2df4308c852240859ada53d216ab69ae" candidate_term: 3 candidate_status { last_received { term: 2 index: 567 } } ignore_live_leader: false dest_uuid: "222b20de5a1145c6b8a11df8374e8790"
I20260812 06:36:20.596688  6919 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" candidate_term: 3 candidate_status { last_received { term: 2 index: 561 } } ignore_live_leader: false dest_uuid: "222b20de5a1145c6b8a11df8374e8790" is_pre_election: true
I20260812 06:36:20.596890  6921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" candidate_term: 3 candidate_status { last_received { term: 2 index: 561 } } ignore_live_leader: false dest_uuid: "222b20de5a1145c6b8a11df8374e8790" is_pre_election: true
I20260812 06:36:20.597244  7725 consensus_queue.cc:1059] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "42895a9bcba64aa484f6509605035d02" member_type: VOTER last_known_addr { host: "127.4.177.4" port: 38181 }, Status: INVALID_TERM, Last received: 2.567, Next index: 568, Last known committed idx: 559, Time since last communication: 0.000s
I20260812 06:36:20.597851  7649 raft_consensus.cc:3055] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [term 2 LEADER]: Stepping down as leader of term 2
I20260812 06:36:20.597903  7649 raft_consensus.cc:740] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 222b20de5a1145c6b8a11df8374e8790, State: Running, Role: LEADER
I20260812 06:36:20.598037  7649 consensus_queue.cc:260] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 561, Committed index: 561, Last appended: 2.567, Last appended by leader: 567, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df4308c852240859ada53d216ab69ae" member_type: VOTER last_known_addr { host: "127.4.177.6" port: 34269 } } peers { permanent_uuid: "42895a9bcba64aa484f6509605035d02" member_type: VOTER last_known_addr { host: "127.4.177.4" port: 38181 } } peers { permanent_uuid: "222b20de5a1145c6b8a11df8374e8790" member_type: VOTER last_known_addr { host: "127.4.177.5" port: 43521 } } peers { permanent_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 40897 } } peers { permanent_uuid: "8d90d61da62549208fe117b4384a469b" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 41539 } } peers { permanent_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" member_type: VOTER last_known_addr { host: "127.4.177.7" port: 46299 } } peers { permanent_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 41057 } }
I20260812 06:36:20.598323  7649 raft_consensus.cc:3060] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [term 2 FOLLOWER]: Advancing to term 4
I20260812 06:36:20.598773  6247 raft_consensus.cc:1240] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b [term 4 FOLLOWER]: Rejecting Update request from peer 222b20de5a1145c6b8a11df8374e8790 for earlier term 2. Current term is 4. Ops: [2.514-2.567]
I20260812 06:36:20.599088  7108 raft_consensus.cc:1240] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [term 4 FOLLOWER]: Rejecting Update request from peer 222b20de5a1145c6b8a11df8374e8790 for earlier term 2. Current term is 4. Ops: []
I20260812 06:36:20.600246  7765 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:57072 (ReqId={client: 3067f3aef2a44cc4bb7f2ccd6acf83db, seq_no=558, attempt_no=0}) took 3729 ms. Trace:
I20260812 06:36:20.600576  7822 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:57072 (ReqId={client: 3067f3aef2a44cc4bb7f2ccd6acf83db, seq_no=557, attempt_no=0}) took 3729 ms. Trace:
I20260812 06:36:20.600649  7822 rpcz_store.cc:276] 0812 06:36:16.870693 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 06:36:16.870841 (+   148us) service_pool.cc:224] Handling call
0812 06:36:20.600550 (+3729709us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0812 06:36:16.871900 (+     0us) write_op.cc:183] PREPARE: starting on tablet 7065bb403e3047dd9a0853c49318ef1f
0812 06:36:16.871964 (+    64us) write_op.cc:432] Acquiring schema lock in shared mode
0812 06:36:16.871967 (+     3us) write_op.cc:435] Acquired schema lock
0812 06:36:16.871969 (+     2us) tablet.cc:662] Decoding operations
0812 06:36:16.872044 (+    75us) write_op.cc:620] Acquiring the partition lock for write op
0812 06:36:16.872050 (+     6us) write_op.cc:641] Partition lock acquired for write op
0812 06:36:16.872053 (+     3us) tablet.cc:685] Acquiring locks for 10 operations
0812 06:36:16.872107 (+    54us) tablet.cc:701] Row locks acquired
0812 06:36:16.872108 (+     1us) write_op.cc:260] PREPARE: finished
0812 06:36:16.872131 (+    23us) write_op.cc:270] Start()
0812 06:36:16.872147 (+    16us) write_op.cc:276] Timestamp: P: 1786516576872062 usec, L: 0
0812 06:36:16.872149 (+     2us) op_driver.cc:348] REPLICATION: starting
0812 06:36:16.872234 (+    85us) log.cc:844] Serialized 696 byte log entry
0812 06:36:20.597071 (+3724837us) op_driver.cc:464] REPLICATION: finished
0812 06:36:20.600101 (+  3030us) write_op.cc:301] APPLY: starting
0812 06:36:20.600130 (+    29us) tablet.cc:1367] starting BulkCheckPresence
0812 06:36:20.600161 (+    31us) tablet.cc:1370] finished BulkCheckPresence
0812 06:36:20.600173 (+    12us) tablet.cc:1372] starting ApplyRowOperation cycle
0812 06:36:20.600347 (+   174us) tablet.cc:1383] finished ApplyRowOperation cycle
0812 06:36:20.600374 (+    27us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0812 06:36:20.600378 (+     4us) write_op.cc:312] APPLY: finished
0812 06:36:20.600433 (+    55us) log.cc:844] Serialized 77 byte log entry
0812 06:36:20.600479 (+    46us) write_op.cc:489] Releasing partition, row and schema locks
0812 06:36:20.600492 (+    13us) write_op.cc:454] Released schema lock
0812 06:36:20.600497 (+     5us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"spinlock_wait_cycles":63360,"child_traces":[["op",{"apply.queue_time_us":2986,"num_ops":10,"prepare.queue_time_us":764,"prepare.run_cpu_time_us":374,"prepare.run_wall_time_us":521,"replication_time_us":3724907,"spinlock_wait_cycles":17152,"thread_start_us":64,"threads_started":1}]]}
I20260812 06:36:20.601840  6479 raft_consensus.cc:1240] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [term 4 LEADER]: Rejecting Update request from peer 222b20de5a1145c6b8a11df8374e8790 for earlier term 2. Current term is 4. Ops: [2.562-2.567]
I20260812 06:36:20.602487  6047 raft_consensus.cc:1240] T 7065bb403e3047dd9a0853c49318ef1f P ec6c8d818e744cdbb6b0d15b321010e0 [term 4 FOLLOWER]: Rejecting Update request from peer 222b20de5a1145c6b8a11df8374e8790 for earlier term 2. Current term is 4. Ops: [2.442-2.567]
W20260812 06:36:20.603581  6848 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:57072: Illegal state: replica 222b20de5a1145c6b8a11df8374e8790 is not leader of this config: current role FOLLOWER
I20260812 06:36:20.600327  7765 rpcz_store.cc:276] 0812 06:36:16.870758 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 06:36:16.870847 (+    89us) service_pool.cc:224] Handling call
0812 06:36:20.600221 (+3729374us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0812 06:36:16.871298 (+     0us) write_op.cc:183] PREPARE: starting on tablet 7065bb403e3047dd9a0853c49318ef1f
0812 06:36:16.871364 (+    66us) write_op.cc:432] Acquiring schema lock in shared mode
0812 06:36:16.871367 (+     3us) write_op.cc:435] Acquired schema lock
0812 06:36:16.871369 (+     2us) tablet.cc:662] Decoding operations
0812 06:36:16.871458 (+    89us) write_op.cc:620] Acquiring the partition lock for write op
0812 06:36:16.871463 (+     5us) write_op.cc:641] Partition lock acquired for write op
0812 06:36:16.871466 (+     3us) tablet.cc:685] Acquiring locks for 10 operations
0812 06:36:16.871531 (+    65us) tablet.cc:701] Row locks acquired
0812 06:36:16.871533 (+     2us) write_op.cc:260] PREPARE: finished
0812 06:36:16.871557 (+    24us) write_op.cc:270] Start()
0812 06:36:16.871573 (+    16us) write_op.cc:276] Timestamp: P: 1786516576871488 usec, L: 0
0812 06:36:16.871574 (+     1us) op_driver.cc:348] REPLICATION: starting
0812 06:36:16.871675 (+   101us) log.cc:844] Serialized 696 byte log entry
0812 06:36:20.597010 (+3725335us) op_driver.cc:464] REPLICATION: finished
0812 06:36:20.599704 (+  2694us) write_op.cc:301] APPLY: starting
0812 06:36:20.599726 (+    22us) tablet.cc:1367] starting BulkCheckPresence
0812 06:36:20.599766 (+    40us) tablet.cc:1370] finished BulkCheckPresence
0812 06:36:20.599778 (+    12us) tablet.cc:1372] starting ApplyRowOperation cycle
0812 06:36:20.599962 (+   184us) tablet.cc:1383] finished ApplyRowOperation cycle
0812 06:36:20.599995 (+    33us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0812 06:36:20.600000 (+     5us) write_op.cc:312] APPLY: finished
0812 06:36:20.600076 (+    76us) log.cc:844] Serialized 77 byte log entry
0812 06:36:20.600121 (+    45us) write_op.cc:489] Releasing partition, row and schema locks
0812 06:36:20.600142 (+    21us) write_op.cc:454] Released schema lock
0812 06:36:20.600148 (+     6us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":2636,"num_ops":10,"prepare.queue_time_us":190,"prepare.run_cpu_time_us":420,"prepare.run_wall_time_us":581,"replication_time_us":3725410}]]}
I20260812 06:36:20.604666  7321 raft_consensus.cc:1240] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08 [term 4 FOLLOWER]: Rejecting Update request from peer 222b20de5a1145c6b8a11df8374e8790 for earlier term 2. Current term is 4. Ops: [2.474-2.567]
I20260812 06:36:20.605314  7020 leader_election.cc:304] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 7 responses out of 7 voters: 3 yes votes; 4 no votes. yes voters: 2df4308c852240859ada53d216ab69ae, 8d90d61da62549208fe117b4384a469b, ec6c8d818e744cdbb6b0d15b321010e0; no voters: 222b20de5a1145c6b8a11df8374e8790, 42895a9bcba64aa484f6509605035d02, 7962f587ccd842ad8a3f6ed9a5d7707e, b037d17b21f8422fa5e41b70eb4dcd08
I20260812 06:36:20.605914  7824 raft_consensus.cc:2749] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [term 4 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
W20260812 06:36:20.606662  6848 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:57072: Illegal state: replica 222b20de5a1145c6b8a11df8374e8790 is not leader of this config: current role FOLLOWER
W20260812 06:36:20.606805  7051 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50492: Illegal state: replica 2df4308c852240859ada53d216ab69ae is not leader of this config: current role FOLLOWER
W20260812 06:36:20.610157  7051 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50492: Illegal state: replica 2df4308c852240859ada53d216ab69ae is not leader of this config: current role FOLLOWER
W20260812 06:36:20.610335  6629 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:42408: Illegal state: replica 42895a9bcba64aa484f6509605035d02 is not leader of this config: current role FOLLOWER
I20260812 06:36:20.611122  7825 raft_consensus.cc:493] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 06:36:20.611215  7825 raft_consensus.cc:515] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df4308c852240859ada53d216ab69ae" member_type: VOTER last_known_addr { host: "127.4.177.6" port: 34269 } } peers { permanent_uuid: "42895a9bcba64aa484f6509605035d02" member_type: VOTER last_known_addr { host: "127.4.177.4" port: 38181 } } peers { permanent_uuid: "222b20de5a1145c6b8a11df8374e8790" member_type: VOTER last_known_addr { host: "127.4.177.5" port: 43521 } } peers { permanent_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 40897 } } peers { permanent_uuid: "8d90d61da62549208fe117b4384a469b" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 41539 } } peers { permanent_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" member_type: VOTER last_known_addr { host: "127.4.177.7" port: 46299 } } peers { permanent_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 41057 } }
I20260812 06:36:20.611877  7825 leader_election.cc:290] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 2df4308c852240859ada53d216ab69ae (127.4.177.6:34269), 222b20de5a1145c6b8a11df8374e8790 (127.4.177.5:43521), 7962f587ccd842ad8a3f6ed9a5d7707e (127.4.177.3:40897), 8d90d61da62549208fe117b4384a469b (127.4.177.2:41539), b037d17b21f8422fa5e41b70eb4dcd08 (127.4.177.7:46299), ec6c8d818e744cdbb6b0d15b321010e0 (127.4.177.1:41057)
I20260812 06:36:20.612303  7108 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "42895a9bcba64aa484f6509605035d02" candidate_term: 5 candidate_status { last_received { term: 2 index: 567 } } ignore_live_leader: false dest_uuid: "2df4308c852240859ada53d216ab69ae" is_pre_election: true
I20260812 06:36:20.612322  6047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "42895a9bcba64aa484f6509605035d02" candidate_term: 5 candidate_status { last_received { term: 2 index: 567 } } ignore_live_leader: false dest_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" is_pre_election: true
I20260812 06:36:20.612442  6047 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P ec6c8d818e744cdbb6b0d15b321010e0 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 42895a9bcba64aa484f6509605035d02 in term 4.
I20260812 06:36:20.612454  7108 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 42895a9bcba64aa484f6509605035d02 in term 4.
I20260812 06:36:20.612475  6247 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "42895a9bcba64aa484f6509605035d02" candidate_term: 5 candidate_status { last_received { term: 2 index: 567 } } ignore_live_leader: false dest_uuid: "8d90d61da62549208fe117b4384a469b" is_pre_election: true
I20260812 06:36:20.612609  6247 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 42895a9bcba64aa484f6509605035d02 in term 4.
I20260812 06:36:20.612898  6597 leader_election.cc:304] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 7 voters: 4 yes votes; 0 no votes. yes voters: 2df4308c852240859ada53d216ab69ae, 42895a9bcba64aa484f6509605035d02, 8d90d61da62549208fe117b4384a469b, ec6c8d818e744cdbb6b0d15b321010e0; no voters: 
I20260812 06:36:20.613132  6479 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "42895a9bcba64aa484f6509605035d02" candidate_term: 5 candidate_status { last_received { term: 2 index: 567 } } ignore_live_leader: false dest_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" is_pre_election: true
I20260812 06:36:20.613288  7825 raft_consensus.cc:2804] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260812 06:36:20.613348  7825 raft_consensus.cc:493] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 06:36:20.613313  6921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "42895a9bcba64aa484f6509605035d02" candidate_term: 5 candidate_status { last_received { term: 2 index: 567 } } ignore_live_leader: false dest_uuid: "222b20de5a1145c6b8a11df8374e8790" is_pre_election: true
I20260812 06:36:20.613421  7825 raft_consensus.cc:3060] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [term 4 FOLLOWER]: Advancing to term 5
I20260812 06:36:20.613435  6921 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 42895a9bcba64aa484f6509605035d02 in term 4.
I20260812 06:36:20.613675  7321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "42895a9bcba64aa484f6509605035d02" candidate_term: 5 candidate_status { last_received { term: 2 index: 567 } } ignore_live_leader: false dest_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" is_pre_election: true
I20260812 06:36:20.613809  7321 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 42895a9bcba64aa484f6509605035d02 in term 4.
I20260812 06:36:20.614634  7825 raft_consensus.cc:515] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df4308c852240859ada53d216ab69ae" member_type: VOTER last_known_addr { host: "127.4.177.6" port: 34269 } } peers { permanent_uuid: "42895a9bcba64aa484f6509605035d02" member_type: VOTER last_known_addr { host: "127.4.177.4" port: 38181 } } peers { permanent_uuid: "222b20de5a1145c6b8a11df8374e8790" member_type: VOTER last_known_addr { host: "127.4.177.5" port: 43521 } } peers { permanent_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 40897 } } peers { permanent_uuid: "8d90d61da62549208fe117b4384a469b" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 41539 } } peers { permanent_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" member_type: VOTER last_known_addr { host: "127.4.177.7" port: 46299 } } peers { permanent_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 41057 } }
I20260812 06:36:20.615545  6921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "42895a9bcba64aa484f6509605035d02" candidate_term: 5 candidate_status { last_received { term: 2 index: 567 } } ignore_live_leader: false dest_uuid: "222b20de5a1145c6b8a11df8374e8790"
I20260812 06:36:20.615660  6921 raft_consensus.cc:3060] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [term 4 FOLLOWER]: Advancing to term 5
I20260812 06:36:20.615722  6247 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "42895a9bcba64aa484f6509605035d02" candidate_term: 5 candidate_status { last_received { term: 2 index: 567 } } ignore_live_leader: false dest_uuid: "8d90d61da62549208fe117b4384a469b"
I20260812 06:36:20.615832  6247 raft_consensus.cc:3060] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b [term 4 FOLLOWER]: Advancing to term 5
I20260812 06:36:20.616274  7321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "42895a9bcba64aa484f6509605035d02" candidate_term: 5 candidate_status { last_received { term: 2 index: 567 } } ignore_live_leader: false dest_uuid: "b037d17b21f8422fa5e41b70eb4dcd08"
I20260812 06:36:20.616396  7321 raft_consensus.cc:3060] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08 [term 4 FOLLOWER]: Advancing to term 5
I20260812 06:36:20.616796  6921 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 42895a9bcba64aa484f6509605035d02 in term 5.
I20260812 06:36:20.616798  6247 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 42895a9bcba64aa484f6509605035d02 in term 5.
I20260812 06:36:20.617035  7108 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "42895a9bcba64aa484f6509605035d02" candidate_term: 5 candidate_status { last_received { term: 2 index: 567 } } ignore_live_leader: false dest_uuid: "2df4308c852240859ada53d216ab69ae"
I20260812 06:36:20.617148  7108 raft_consensus.cc:3060] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [term 4 FOLLOWER]: Advancing to term 5
I20260812 06:36:20.617513  7321 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 42895a9bcba64aa484f6509605035d02 in term 5.
I20260812 06:36:20.617972  7825 leader_election.cc:290] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [CANDIDATE]: Term 5 election: Requested vote from peers 2df4308c852240859ada53d216ab69ae (127.4.177.6:34269), 222b20de5a1145c6b8a11df8374e8790 (127.4.177.5:43521), 7962f587ccd842ad8a3f6ed9a5d7707e (127.4.177.3:40897), 8d90d61da62549208fe117b4384a469b (127.4.177.2:41539), b037d17b21f8422fa5e41b70eb4dcd08 (127.4.177.7:46299), ec6c8d818e744cdbb6b0d15b321010e0 (127.4.177.1:41057)
I20260812 06:36:20.618075  7108 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 42895a9bcba64aa484f6509605035d02 in term 5.
I20260812 06:36:20.618379  6479 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "42895a9bcba64aa484f6509605035d02" candidate_term: 5 candidate_status { last_received { term: 2 index: 567 } } ignore_live_leader: false dest_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e"
I20260812 06:36:20.618558  6597 leader_election.cc:304] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 7 voters: 4 yes votes; 0 no votes. yes voters: 2df4308c852240859ada53d216ab69ae, 42895a9bcba64aa484f6509605035d02, 8d90d61da62549208fe117b4384a469b, b037d17b21f8422fa5e41b70eb4dcd08; no voters: 
I20260812 06:36:20.618738  7825 raft_consensus.cc:2804] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [term 5 FOLLOWER]: Leader election won for term 5
I20260812 06:36:20.618723  6047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "42895a9bcba64aa484f6509605035d02" candidate_term: 5 candidate_status { last_received { term: 2 index: 567 } } ignore_live_leader: false dest_uuid: "ec6c8d818e744cdbb6b0d15b321010e0"
I20260812 06:36:20.618856  6047 raft_consensus.cc:3060] T 7065bb403e3047dd9a0853c49318ef1f P ec6c8d818e744cdbb6b0d15b321010e0 [term 4 FOLLOWER]: Advancing to term 5
I20260812 06:36:20.618950  7825 raft_consensus.cc:697] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [term 5 LEADER]: Becoming Leader. State: Replica: 42895a9bcba64aa484f6509605035d02, State: Running, Role: LEADER
I20260812 06:36:20.619320  7825 consensus_queue.cc:237] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 559, Committed index: 559, Last appended: 2.567, Last appended by leader: 567, Current term: 5, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df4308c852240859ada53d216ab69ae" member_type: VOTER last_known_addr { host: "127.4.177.6" port: 34269 } } peers { permanent_uuid: "42895a9bcba64aa484f6509605035d02" member_type: VOTER last_known_addr { host: "127.4.177.4" port: 38181 } } peers { permanent_uuid: "222b20de5a1145c6b8a11df8374e8790" member_type: VOTER last_known_addr { host: "127.4.177.5" port: 43521 } } peers { permanent_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 40897 } } peers { permanent_uuid: "8d90d61da62549208fe117b4384a469b" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 41539 } } peers { permanent_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" member_type: VOTER last_known_addr { host: "127.4.177.7" port: 46299 } } peers { permanent_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 41057 } }
I20260812 06:36:20.619887  6047 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P ec6c8d818e744cdbb6b0d15b321010e0 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 42895a9bcba64aa484f6509605035d02 in term 5.
I20260812 06:36:20.620796  7321 raft_consensus.cc:1240] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08 [term 5 FOLLOWER]: Rejecting Update request from peer 7962f587ccd842ad8a3f6ed9a5d7707e for earlier term 4. Current term is 5. Ops: []
I20260812 06:36:20.620996  6247 raft_consensus.cc:1240] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b [term 5 FOLLOWER]: Rejecting Update request from peer 7962f587ccd842ad8a3f6ed9a5d7707e for earlier term 4. Current term is 5. Ops: []
I20260812 06:36:20.621194  6047 raft_consensus.cc:1240] T 7065bb403e3047dd9a0853c49318ef1f P ec6c8d818e744cdbb6b0d15b321010e0 [term 5 FOLLOWER]: Rejecting Update request from peer 7962f587ccd842ad8a3f6ed9a5d7707e for earlier term 4. Current term is 5. Ops: []
I20260812 06:36:20.621431  7810 consensus_queue.cc:1059] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8d90d61da62549208fe117b4384a469b" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 41539 }, Status: INVALID_TERM, Last received: 0.0, Next index: 562, Last known committed idx: 507, Time since last communication: 0.000s
I20260812 06:36:20.621493  7108 raft_consensus.cc:1240] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [term 5 FOLLOWER]: Rejecting Update request from peer 7962f587ccd842ad8a3f6ed9a5d7707e for earlier term 4. Current term is 5. Ops: []
I20260812 06:36:20.621712  7810 consensus_queue.cc:1059] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" member_type: VOTER last_known_addr { host: "127.4.177.7" port: 46299 }, Status: INVALID_TERM, Last received: 0.0, Next index: 562, Last known committed idx: 469, Time since last communication: 0.000s
I20260812 06:36:20.621837  7810 consensus_queue.cc:1059] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 41057 }, Status: INVALID_TERM, Last received: 0.0, Next index: 562, Last known committed idx: 439, Time since last communication: 0.000s
I20260812 06:36:20.621977  7810 raft_consensus.cc:3055] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [term 4 LEADER]: Stepping down as leader of term 4
I20260812 06:36:20.622020  7810 raft_consensus.cc:740] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 7962f587ccd842ad8a3f6ed9a5d7707e, State: Running, Role: LEADER
I20260812 06:36:20.622542  6703 raft_consensus.cc:1240] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [term 5 LEADER]: Rejecting Update request from peer 7962f587ccd842ad8a3f6ed9a5d7707e for earlier term 4. Current term is 5. Ops: []
I20260812 06:36:20.622712  6921 raft_consensus.cc:1240] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [term 5 FOLLOWER]: Rejecting Update request from peer 7962f587ccd842ad8a3f6ed9a5d7707e for earlier term 4. Current term is 5. Ops: []
I20260812 06:36:20.623775  7810 consensus_queue.cc:260] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 559, Committed index: 559, Last appended: 4.564, Last appended by leader: 564, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df4308c852240859ada53d216ab69ae" member_type: VOTER last_known_addr { host: "127.4.177.6" port: 34269 } } peers { permanent_uuid: "42895a9bcba64aa484f6509605035d02" member_type: VOTER last_known_addr { host: "127.4.177.4" port: 38181 } } peers { permanent_uuid: "222b20de5a1145c6b8a11df8374e8790" member_type: VOTER last_known_addr { host: "127.4.177.5" port: 43521 } } peers { permanent_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 40897 } } peers { permanent_uuid: "8d90d61da62549208fe117b4384a469b" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 41539 } } peers { permanent_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" member_type: VOTER last_known_addr { host: "127.4.177.7" port: 46299 } } peers { permanent_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 41057 } }
I20260812 06:36:20.623872  5812 catalog_manager.cc:5719] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 reported cstate change: term changed from 4 to 5, leader changed from 7962f587ccd842ad8a3f6ed9a5d7707e (127.4.177.3) to 42895a9bcba64aa484f6509605035d02 (127.4.177.4). New cstate: current_term: 5 leader_uuid: "42895a9bcba64aa484f6509605035d02" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df4308c852240859ada53d216ab69ae" member_type: VOTER last_known_addr { host: "127.4.177.6" port: 34269 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "42895a9bcba64aa484f6509605035d02" member_type: VOTER last_known_addr { host: "127.4.177.4" port: 38181 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "222b20de5a1145c6b8a11df8374e8790" member_type: VOTER last_known_addr { host: "127.4.177.5" port: 43521 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 40897 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8d90d61da62549208fe117b4384a469b" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 41539 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" member_type: VOTER last_known_addr { host: "127.4.177.7" port: 46299 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 41057 } health_report { overall_health: UNKNOWN } } }
I20260812 06:36:20.624214  7837 consensus_queue.cc:1059] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2df4308c852240859ada53d216ab69ae" member_type: VOTER last_known_addr { host: "127.4.177.6" port: 34269 }, Status: INVALID_TERM, Last received: 0.0, Next index: 562, Last known committed idx: 559, Time since last communication: 0.000s
I20260812 06:36:20.624641  7837 consensus_queue.cc:1243] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [NON_LEADER]: Peer 2df4308c852240859ada53d216ab69ae log is divergent from this leader: its last log entry 2.567 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 559
I20260812 06:36:20.624745  7836 consensus_queue.cc:1059] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "222b20de5a1145c6b8a11df8374e8790" member_type: VOTER last_known_addr { host: "127.4.177.5" port: 43521 }, Status: INVALID_TERM, Last received: 0.0, Next index: 562, Last known committed idx: 561, Time since last communication: 0.000s
I20260812 06:36:20.625301  7836 consensus_queue.cc:1243] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [NON_LEADER]: Peer 222b20de5a1145c6b8a11df8374e8790 log is divergent from this leader: its last log entry 2.567 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 561
I20260812 06:36:20.625419  7839 consensus_queue.cc:1059] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "42895a9bcba64aa484f6509605035d02" member_type: VOTER last_known_addr { host: "127.4.177.4" port: 38181 }, Status: INVALID_TERM, Last received: 0.0, Next index: 562, Last known committed idx: 559, Time since last communication: 0.000s
I20260812 06:36:20.625558  7839 consensus_queue.cc:1243] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [NON_LEADER]: Peer 42895a9bcba64aa484f6509605035d02 log is divergent from this leader: its last log entry 5.568 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 559
I20260812 06:36:20.626389  7810 raft_consensus.cc:3060] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [term 4 FOLLOWER]: Advancing to term 5
I20260812 06:36:20.627471  7321 raft_consensus.cc:1240] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08 [term 5 FOLLOWER]: Rejecting Update request from peer 7962f587ccd842ad8a3f6ed9a5d7707e for earlier term 4. Current term is 5. Ops: [2.474-4.564]
I20260812 06:36:20.627729  6247 raft_consensus.cc:1240] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b [term 5 FOLLOWER]: Rejecting Update request from peer 7962f587ccd842ad8a3f6ed9a5d7707e for earlier term 4. Current term is 5. Ops: [2.514-4.564]
I20260812 06:36:20.629818  6047 raft_consensus.cc:1240] T 7065bb403e3047dd9a0853c49318ef1f P ec6c8d818e744cdbb6b0d15b321010e0 [term 5 FOLLOWER]: Rejecting Update request from peer 7962f587ccd842ad8a3f6ed9a5d7707e for earlier term 4. Current term is 5. Ops: [2.442-4.564]
I20260812 06:36:21.025744  7321 raft_consensus.cc:1275] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08 [term 5 FOLLOWER]: Refusing update from remote peer 42895a9bcba64aa484f6509605035d02: Log matching property violated. Preceding OpId in replica: term: 2 index: 473. Preceding OpId from leader: term: 5 index: 568. (index mismatch)
I20260812 06:36:21.026381  7825 consensus_queue.cc:1048] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" member_type: VOTER last_known_addr { host: "127.4.177.7" port: 46299 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 568, Last known committed idx: 469, Time since last communication: 0.000s
I20260812 06:36:21.048765  6921 raft_consensus.cc:1275] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [term 5 FOLLOWER]: Refusing update from remote peer 42895a9bcba64aa484f6509605035d02: Log matching property violated. Preceding OpId in replica: term: 2 index: 567. Preceding OpId from leader: term: 5 index: 568. (index mismatch)
I20260812 06:36:21.049373  7825 consensus_queue.cc:1048] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [LEADER]: Connected to new peer: Peer: permanent_uuid: "222b20de5a1145c6b8a11df8374e8790" member_type: VOTER last_known_addr { host: "127.4.177.5" port: 43521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 568, Last known committed idx: 561, Time since last communication: 0.000s
I20260812 06:36:21.089031  7108 raft_consensus.cc:1275] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [term 5 FOLLOWER]: Refusing update from remote peer 42895a9bcba64aa484f6509605035d02: Log matching property violated. Preceding OpId in replica: term: 2 index: 567. Preceding OpId from leader: term: 5 index: 568. (index mismatch)
I20260812 06:36:21.089592  7825 consensus_queue.cc:1048] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2df4308c852240859ada53d216ab69ae" member_type: VOTER last_known_addr { host: "127.4.177.6" port: 34269 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 568, Last known committed idx: 559, Time since last communication: 0.000s
I20260812 06:36:21.094439  6479 raft_consensus.cc:1275] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [term 5 FOLLOWER]: Refusing update from remote peer 42895a9bcba64aa484f6509605035d02: Log matching property violated. Preceding OpId in replica: term: 4 index: 564. Preceding OpId from leader: term: 5 index: 568. (index mismatch)
I20260812 06:36:21.094797  6047 raft_consensus.cc:1275] T 7065bb403e3047dd9a0853c49318ef1f P ec6c8d818e744cdbb6b0d15b321010e0 [term 5 FOLLOWER]: Refusing update from remote peer 42895a9bcba64aa484f6509605035d02: Log matching property violated. Preceding OpId in replica: term: 2 index: 441. Preceding OpId from leader: term: 5 index: 568. (index mismatch)
I20260812 06:36:21.094851  6247 raft_consensus.cc:1275] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b [term 5 FOLLOWER]: Refusing update from remote peer 42895a9bcba64aa484f6509605035d02: Log matching property violated. Preceding OpId in replica: term: 2 index: 513. Preceding OpId from leader: term: 5 index: 568. (index mismatch)
I20260812 06:36:21.095325  7822 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:57072 (ReqId={client: 3067f3aef2a44cc4bb7f2ccd6acf83db, seq_no=560, attempt_no=0}) took 4210 ms. Trace:
I20260812 06:36:21.095403  7822 rpcz_store.cc:276] 0812 06:36:16.884474 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 06:36:16.884540 (+    66us) service_pool.cc:224] Handling call
0812 06:36:21.095310 (+4210770us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0812 06:36:16.885181 (+     0us) write_op.cc:183] PREPARE: starting on tablet 7065bb403e3047dd9a0853c49318ef1f
0812 06:36:16.885281 (+   100us) write_op.cc:432] Acquiring schema lock in shared mode
0812 06:36:16.885285 (+     4us) write_op.cc:435] Acquired schema lock
0812 06:36:16.885286 (+     1us) tablet.cc:662] Decoding operations
0812 06:36:16.885376 (+    90us) write_op.cc:620] Acquiring the partition lock for write op
0812 06:36:16.885381 (+     5us) write_op.cc:641] Partition lock acquired for write op
0812 06:36:16.885384 (+     3us) tablet.cc:685] Acquiring locks for 10 operations
0812 06:36:16.885442 (+    58us) tablet.cc:701] Row locks acquired
0812 06:36:16.885443 (+     1us) write_op.cc:260] PREPARE: finished
0812 06:36:16.885467 (+    24us) write_op.cc:270] Start()
0812 06:36:16.885485 (+    18us) write_op.cc:276] Timestamp: P: 1786516576885396 usec, L: 0
0812 06:36:16.885486 (+     1us) op_driver.cc:348] REPLICATION: starting
0812 06:36:16.885591 (+   105us) log.cc:844] Serialized 676 byte log entry
0812 06:36:21.094118 (+4208527us) op_driver.cc:464] REPLICATION: finished
0812 06:36:21.094842 (+   724us) write_op.cc:301] APPLY: starting
0812 06:36:21.094854 (+    12us) tablet.cc:1367] starting BulkCheckPresence
0812 06:36:21.094889 (+    35us) tablet.cc:1370] finished BulkCheckPresence
0812 06:36:21.094920 (+    31us) tablet.cc:1372] starting ApplyRowOperation cycle
0812 06:36:21.095089 (+   169us) tablet.cc:1383] finished ApplyRowOperation cycle
0812 06:36:21.095120 (+    31us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0812 06:36:21.095124 (+     4us) write_op.cc:312] APPLY: finished
0812 06:36:21.095191 (+    67us) log.cc:844] Serialized 77 byte log entry
0812 06:36:21.095227 (+    36us) write_op.cc:489] Releasing partition, row and schema locks
0812 06:36:21.095243 (+    16us) write_op.cc:454] Released schema lock
0812 06:36:21.095248 (+     5us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":679,"num_ops":10,"prepare.queue_time_us":315,"prepare.run_cpu_time_us":507,"prepare.run_wall_time_us":506,"raft.queue_time_us":61,"raft.run_cpu_time_us":152,"raft.run_wall_time_us":152,"replication_time_us":4208606}]]}
I20260812 06:36:21.095587  7765 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:57072 (ReqId={client: 3067f3aef2a44cc4bb7f2ccd6acf83db, seq_no=563, attempt_no=0}) took 4210 ms. Trace:
I20260812 06:36:21.095620  7825 consensus_queue.cc:1048] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 40897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 568, Last known committed idx: 559, Time since last communication: 0.000s
I20260812 06:36:21.095728  7825 consensus_queue.cc:1243] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [LEADER]: Peer 7962f587ccd842ad8a3f6ed9a5d7707e log is divergent from this leader: its last log entry 4.564 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 559
I20260812 06:36:21.095665  7765 rpcz_store.cc:276] 0812 06:36:16.884670 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 06:36:16.884708 (+    38us) service_pool.cc:224] Handling call
0812 06:36:21.095574 (+4210866us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0812 06:36:16.885714 (+     0us) write_op.cc:183] PREPARE: starting on tablet 7065bb403e3047dd9a0853c49318ef1f
0812 06:36:16.885787 (+    73us) write_op.cc:432] Acquiring schema lock in shared mode
0812 06:36:16.885791 (+     4us) write_op.cc:435] Acquired schema lock
0812 06:36:16.885792 (+     1us) tablet.cc:662] Decoding operations
0812 06:36:16.885887 (+    95us) write_op.cc:620] Acquiring the partition lock for write op
0812 06:36:16.885894 (+     7us) write_op.cc:641] Partition lock acquired for write op
0812 06:36:16.885896 (+     2us) tablet.cc:685] Acquiring locks for 10 operations
0812 06:36:16.885967 (+    71us) tablet.cc:701] Row locks acquired
0812 06:36:16.885970 (+     3us) write_op.cc:260] PREPARE: finished
0812 06:36:16.886002 (+    32us) write_op.cc:270] Start()
0812 06:36:16.886024 (+    22us) write_op.cc:276] Timestamp: P: 1786516576885928 usec, L: 0
0812 06:36:16.886026 (+     2us) op_driver.cc:348] REPLICATION: starting
0812 06:36:16.886127 (+   101us) log.cc:844] Serialized 686 byte log entry
0812 06:36:21.094189 (+4208062us) op_driver.cc:464] REPLICATION: finished
0812 06:36:21.095062 (+   873us) write_op.cc:301] APPLY: starting
0812 06:36:21.095078 (+    16us) tablet.cc:1367] starting BulkCheckPresence
0812 06:36:21.095113 (+    35us) tablet.cc:1370] finished BulkCheckPresence
0812 06:36:21.095130 (+    17us) tablet.cc:1372] starting ApplyRowOperation cycle
0812 06:36:21.095333 (+   203us) tablet.cc:1383] finished ApplyRowOperation cycle
0812 06:36:21.095365 (+    32us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0812 06:36:21.095370 (+     5us) write_op.cc:312] APPLY: finished
0812 06:36:21.095429 (+    59us) log.cc:844] Serialized 77 byte log entry
0812 06:36:21.095479 (+    50us) write_op.cc:489] Releasing partition, row and schema locks
0812 06:36:21.095497 (+    18us) write_op.cc:454] Released schema lock
0812 06:36:21.095505 (+     8us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":821,"num_ops":10,"prepare.queue_time_us":724,"prepare.run_cpu_time_us":485,"prepare.run_wall_time_us":484,"replication_time_us":4208149}]]}
I20260812 06:36:21.096021  7825 consensus_queue.cc:1048] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8d90d61da62549208fe117b4384a469b" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 41539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 568, Last known committed idx: 507, Time since last communication: 0.000s
I20260812 06:36:21.096212  7822 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:57072 (ReqId={client: 3067f3aef2a44cc4bb7f2ccd6acf83db, seq_no=562, attempt_no=0}) took 4211 ms. Trace:
I20260812 06:36:21.096374  7765 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:57072 (ReqId={client: 3067f3aef2a44cc4bb7f2ccd6acf83db, seq_no=559, attempt_no=0}) took 4212 ms. Trace:
I20260812 06:36:21.096459  7765 rpcz_store.cc:276] 0812 06:36:16.884222 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 06:36:16.884873 (+   651us) service_pool.cc:224] Handling call
0812 06:36:21.096363 (+4211490us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0812 06:36:16.886715 (+     0us) write_op.cc:183] PREPARE: starting on tablet 7065bb403e3047dd9a0853c49318ef1f
0812 06:36:16.886785 (+    70us) write_op.cc:432] Acquiring schema lock in shared mode
0812 06:36:16.886788 (+     3us) write_op.cc:435] Acquired schema lock
0812 06:36:16.886789 (+     1us) tablet.cc:662] Decoding operations
0812 06:36:16.886872 (+    83us) write_op.cc:620] Acquiring the partition lock for write op
0812 06:36:16.886877 (+     5us) write_op.cc:641] Partition lock acquired for write op
0812 06:36:16.886880 (+     3us) tablet.cc:685] Acquiring locks for 10 operations
0812 06:36:16.886943 (+    63us) tablet.cc:701] Row locks acquired
0812 06:36:16.886943 (+     0us) write_op.cc:260] PREPARE: finished
0812 06:36:16.886965 (+    22us) write_op.cc:270] Start()
0812 06:36:16.886979 (+    14us) write_op.cc:276] Timestamp: P: 1786516576886895 usec, L: 0
0812 06:36:16.886981 (+     2us) op_driver.cc:348] REPLICATION: starting
0812 06:36:16.887061 (+    80us) log.cc:844] Serialized 696 byte log entry
0812 06:36:21.094372 (+4207311us) op_driver.cc:464] REPLICATION: finished
0812 06:36:21.095995 (+  1623us) write_op.cc:301] APPLY: starting
0812 06:36:21.096006 (+    11us) tablet.cc:1367] starting BulkCheckPresence
0812 06:36:21.096033 (+    27us) tablet.cc:1370] finished BulkCheckPresence
0812 06:36:21.096047 (+    14us) tablet.cc:1372] starting ApplyRowOperation cycle
0812 06:36:21.096199 (+   152us) tablet.cc:1383] finished ApplyRowOperation cycle
0812 06:36:21.096224 (+    25us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0812 06:36:21.096228 (+     4us) write_op.cc:312] APPLY: finished
0812 06:36:21.096273 (+    45us) log.cc:844] Serialized 77 byte log entry
0812 06:36:21.096302 (+    29us) write_op.cc:489] Releasing partition, row and schema locks
0812 06:36:21.096313 (+    11us) write_op.cc:454] Released schema lock
0812 06:36:21.096317 (+     4us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"spinlock_wait_cycles":11648,"child_traces":[["op",{"apply.queue_time_us":1594,"num_ops":10,"prepare.queue_time_us":1559,"prepare.run_cpu_time_us":416,"prepare.run_wall_time_us":415,"replication_time_us":4207378,"thread_start_us":47,"threads_started":1}]]}
I20260812 06:36:21.097254  7765 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:57072 (ReqId={client: 3067f3aef2a44cc4bb7f2ccd6acf83db, seq_no=564, attempt_no=0}) took 4212 ms. Trace:
I20260812 06:36:21.097321  7765 rpcz_store.cc:276] 0812 06:36:16.885207 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 06:36:16.885238 (+    31us) service_pool.cc:224] Handling call
0812 06:36:21.097245 (+4212007us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0812 06:36:16.887159 (+     0us) write_op.cc:183] PREPARE: starting on tablet 7065bb403e3047dd9a0853c49318ef1f
0812 06:36:16.887234 (+    75us) write_op.cc:432] Acquiring schema lock in shared mode
0812 06:36:16.887239 (+     5us) write_op.cc:435] Acquired schema lock
0812 06:36:16.887240 (+     1us) tablet.cc:662] Decoding operations
0812 06:36:16.887324 (+    84us) write_op.cc:620] Acquiring the partition lock for write op
0812 06:36:16.887330 (+     6us) write_op.cc:641] Partition lock acquired for write op
0812 06:36:16.887333 (+     3us) tablet.cc:685] Acquiring locks for 10 operations
0812 06:36:16.887390 (+    57us) tablet.cc:701] Row locks acquired
0812 06:36:16.887391 (+     1us) write_op.cc:260] PREPARE: finished
0812 06:36:16.887412 (+    21us) write_op.cc:270] Start()
0812 06:36:16.887434 (+    22us) write_op.cc:276] Timestamp: P: 1786516576887343 usec, L: 0
0812 06:36:16.887435 (+     1us) op_driver.cc:348] REPLICATION: starting
0812 06:36:16.887513 (+    78us) log.cc:844] Serialized 696 byte log entry
0812 06:36:21.094477 (+4206964us) op_driver.cc:464] REPLICATION: finished
0812 06:36:21.096766 (+  2289us) write_op.cc:301] APPLY: starting
0812 06:36:21.096774 (+     8us) tablet.cc:1367] starting BulkCheckPresence
0812 06:36:21.096799 (+    25us) tablet.cc:1370] finished BulkCheckPresence
0812 06:36:21.096811 (+    12us) tablet.cc:1372] starting ApplyRowOperation cycle
0812 06:36:21.097016 (+   205us) tablet.cc:1383] finished ApplyRowOperation cycle
0812 06:36:21.097051 (+    35us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0812 06:36:21.097057 (+     6us) write_op.cc:312] APPLY: finished
0812 06:36:21.097124 (+    67us) log.cc:844] Serialized 77 byte log entry
0812 06:36:21.097176 (+    52us) write_op.cc:489] Releasing partition, row and schema locks
0812 06:36:21.097190 (+    14us) write_op.cc:454] Released schema lock
0812 06:36:21.097194 (+     4us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":2263,"num_ops":10,"prepare.queue_time_us":1761,"prepare.run_cpu_time_us":419,"prepare.run_wall_time_us":419,"replication_time_us":4207013,"spinlock_wait_cycles":768}]]}
I20260812 06:36:21.098047  7765 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:57072 (ReqId={client: 3067f3aef2a44cc4bb7f2ccd6acf83db, seq_no=561, attempt_no=0}) took 4213 ms. Trace:
I20260812 06:36:21.102589  6479 raft_consensus.cc:1217] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [term 5 FOLLOWER]: Deduplicated request from leader. Original: 2.559->[2.560-5.568]   Dedup: 2.561->[2.562-5.568]
I20260812 06:36:21.102751  6479 pending_rounds.cc:85] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e: Aborting all ops after (but not including) 561
I20260812 06:36:21.102816  6479 pending_rounds.cc:107] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e: Aborting uncommitted NO_OP operation due to leader change: 4.562
I20260812 06:36:21.102851  6479 raft_consensus.cc:2889] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260812 06:36:21.102896  6479 pending_rounds.cc:107] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e: Aborting uncommitted WRITE_OP operation due to leader change: 4.563
I20260812 06:36:21.096280  7822 rpcz_store.cc:276] 0812 06:36:16.884618 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 06:36:16.884893 (+   275us) service_pool.cc:224] Handling call
0812 06:36:21.096201 (+4211308us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0812 06:36:16.886228 (+     0us) write_op.cc:183] PREPARE: starting on tablet 7065bb403e3047dd9a0853c49318ef1f
0812 06:36:16.886321 (+    93us) write_op.cc:432] Acquiring schema lock in shared mode
0812 06:36:16.886327 (+     6us) write_op.cc:435] Acquired schema lock
0812 06:36:16.886328 (+     1us) tablet.cc:662] Decoding operations
0812 06:36:16.886414 (+    86us) write_op.cc:620] Acquiring the partition lock for write op
0812 06:36:16.886420 (+     6us) write_op.cc:641] Partition lock acquired for write op
0812 06:36:16.886432 (+    12us) tablet.cc:685] Acquiring locks for 10 operations
0812 06:36:16.886495 (+    63us) tablet.cc:701] Row locks acquired
0812 06:36:16.886496 (+     1us) write_op.cc:260] PREPARE: finished
0812 06:36:16.886518 (+    22us) write_op.cc:270] Start()
0812 06:36:16.886533 (+    15us) write_op.cc:276] Timestamp: P: 1786516576886448 usec, L: 0
0812 06:36:16.886534 (+     1us) op_driver.cc:348] REPLICATION: starting
0812 06:36:16.886617 (+    83us) log.cc:844] Serialized 696 byte log entry
0812 06:36:21.094253 (+4207636us) op_driver.cc:464] REPLICATION: finished
0812 06:36:21.095797 (+  1544us) write_op.cc:301] APPLY: starting
0812 06:36:21.095805 (+     8us) tablet.cc:1367] starting BulkCheckPresence
0812 06:36:21.095828 (+    23us) tablet.cc:1370] finished BulkCheckPresence
0812 06:36:21.095839 (+    11us) tablet.cc:1372] starting ApplyRowOperation cycle
0812 06:36:21.096022 (+   183us) tablet.cc:1383] finished ApplyRowOperation cycle
0812 06:36:21.096046 (+    24us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0812 06:36:21.096050 (+     4us) write_op.cc:312] APPLY: finished
0812 06:36:21.096098 (+    48us) log.cc:844] Serialized 77 byte log entry
0812 06:36:21.096131 (+    33us) write_op.cc:489] Releasing partition, row and schema locks
0812 06:36:21.096143 (+    12us) write_op.cc:454] Released schema lock
0812 06:36:21.096147 (+     4us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"spinlock_wait_cycles":28544,"child_traces":[["op",{"apply.queue_time_us":1519,"num_ops":10,"prepare.queue_time_us":1089,"prepare.run_cpu_time_us":459,"prepare.run_wall_time_us":458,"replication_time_us":4207708,"thread_start_us":73,"threads_started":1}]]}
W20260812 06:36:21.103040  6479 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:33412: Aborted: Op aborted by new leader
I20260812 06:36:21.098110  7765 rpcz_store.cc:276] 0812 06:36:16.884564 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 06:36:16.889134 (+  4570us) service_pool.cc:224] Handling call
0812 06:36:21.098037 (+4208903us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0812 06:36:16.889438 (+     0us) write_op.cc:183] PREPARE: starting on tablet 7065bb403e3047dd9a0853c49318ef1f
0812 06:36:16.889520 (+    82us) write_op.cc:432] Acquiring schema lock in shared mode
0812 06:36:16.889522 (+     2us) write_op.cc:435] Acquired schema lock
0812 06:36:16.889523 (+     1us) tablet.cc:662] Decoding operations
0812 06:36:16.889588 (+    65us) write_op.cc:620] Acquiring the partition lock for write op
0812 06:36:16.889592 (+     4us) write_op.cc:641] Partition lock acquired for write op
0812 06:36:16.889594 (+     2us) tablet.cc:685] Acquiring locks for 10 operations
0812 06:36:16.889640 (+    46us) tablet.cc:701] Row locks acquired
0812 06:36:16.889641 (+     1us) write_op.cc:260] PREPARE: finished
0812 06:36:16.889665 (+    24us) write_op.cc:270] Start()
0812 06:36:16.889681 (+    16us) write_op.cc:276] Timestamp: P: 1786516576889593 usec, L: 0
0812 06:36:16.889682 (+     1us) op_driver.cc:348] REPLICATION: starting
0812 06:36:16.889770 (+    88us) log.cc:844] Serialized 696 byte log entry
0812 06:36:21.094508 (+4204738us) op_driver.cc:464] REPLICATION: finished
0812 06:36:21.097640 (+  3132us) write_op.cc:301] APPLY: starting
0812 06:36:21.097648 (+     8us) tablet.cc:1367] starting BulkCheckPresence
0812 06:36:21.097672 (+    24us) tablet.cc:1370] finished BulkCheckPresence
0812 06:36:21.097682 (+    10us) tablet.cc:1372] starting ApplyRowOperation cycle
0812 06:36:21.097846 (+   164us) tablet.cc:1383] finished ApplyRowOperation cycle
0812 06:36:21.097869 (+    23us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0812 06:36:21.097873 (+     4us) write_op.cc:312] APPLY: finished
0812 06:36:21.097936 (+    63us) log.cc:844] Serialized 77 byte log entry
0812 06:36:21.097964 (+    28us) write_op.cc:489] Releasing partition, row and schema locks
0812 06:36:21.097987 (+    23us) write_op.cc:454] Released schema lock
0812 06:36:21.097991 (+     4us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":3109,"num_ops":10,"prepare.queue_time_us":49,"prepare.run_cpu_time_us":426,"prepare.run_wall_time_us":426,"replication_time_us":4204816}]]}
I20260812 06:36:21.103307  6479 pending_rounds.cc:107] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e: Aborting uncommitted WRITE_OP operation due to leader change: 4.564
W20260812 06:36:21.103440  6479 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:33412: Aborted: Op aborted by new leader
W20260812 06:36:21.105257  6406 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:33412: Illegal state: replica 7962f587ccd842ad8a3f6ed9a5d7707e is not leader of this config: current role FOLLOWER
W20260812 06:36:21.105440  6403 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:33412: Illegal state: replica 7962f587ccd842ad8a3f6ed9a5d7707e is not leader of this config: current role FOLLOWER
W20260812 06:36:21.105741  6407 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:33412: Illegal state: replica 7962f587ccd842ad8a3f6ed9a5d7707e is not leader of this config: current role FOLLOWER
W20260812 06:36:21.106280  6404 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:33412: Illegal state: replica 7962f587ccd842ad8a3f6ed9a5d7707e is not leader of this config: current role FOLLOWER
W20260812 06:36:21.107196  6404 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:33412: Illegal state: replica 7962f587ccd842ad8a3f6ed9a5d7707e is not leader of this config: current role FOLLOWER
W20260812 06:36:21.107880  6404 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:33412: Illegal state: replica 7962f587ccd842ad8a3f6ed9a5d7707e is not leader of this config: current role FOLLOWER
I20260812 06:36:21.107882  7830 consensus_queue.cc:1048] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 41057 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 568, Last known committed idx: 439, Time since last communication: 0.000s
W20260812 06:36:21.109750  7051 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50492: Illegal state: replica 2df4308c852240859ada53d216ab69ae is not leader of this config: current role FOLLOWER
W20260812 06:36:21.117085  7060 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50492: Illegal state: replica 2df4308c852240859ada53d216ab69ae is not leader of this config: current role FOLLOWER
W20260812 06:36:21.117599  7050 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50492: Illegal state: replica 2df4308c852240859ada53d216ab69ae is not leader of this config: current role FOLLOWER
W20260812 06:36:21.118052  7059 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50492: Illegal state: replica 2df4308c852240859ada53d216ab69ae is not leader of this config: current role FOLLOWER
W20260812 06:36:21.119007  7051 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50492: Illegal state: replica 2df4308c852240859ada53d216ab69ae is not leader of this config: current role FOLLOWER
W20260812 06:36:21.127307  6185 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:42054: Illegal state: replica 8d90d61da62549208fe117b4384a469b is not leader of this config: current role FOLLOWER
W20260812 06:36:21.128223  6178 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:42054: Illegal state: replica 8d90d61da62549208fe117b4384a469b is not leader of this config: current role FOLLOWER
W20260812 06:36:21.128435  6183 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:42054: Illegal state: replica 8d90d61da62549208fe117b4384a469b is not leader of this config: current role FOLLOWER
W20260812 06:36:21.128790  6179 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:42054: Illegal state: replica 8d90d61da62549208fe117b4384a469b is not leader of this config: current role FOLLOWER
W20260812 06:36:21.129120  6182 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:42054: Illegal state: replica 8d90d61da62549208fe117b4384a469b is not leader of this config: current role FOLLOWER
W20260812 06:36:21.127314  6184 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:42054: Illegal state: replica 8d90d61da62549208fe117b4384a469b is not leader of this config: current role FOLLOWER
W20260812 06:36:21.129411  6180 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:42054: Illegal state: replica 8d90d61da62549208fe117b4384a469b is not leader of this config: current role FOLLOWER
W20260812 06:36:21.130519  6184 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:42054: Illegal state: replica 8d90d61da62549208fe117b4384a469b is not leader of this config: current role FOLLOWER
W20260812 06:36:21.158257  7259 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:58454: Illegal state: replica b037d17b21f8422fa5e41b70eb4dcd08 is not leader of this config: current role FOLLOWER
W20260812 06:36:21.159147  7255 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:58454: Illegal state: replica b037d17b21f8422fa5e41b70eb4dcd08 is not leader of this config: current role FOLLOWER
W20260812 06:36:21.159655  7254 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:58454: Illegal state: replica b037d17b21f8422fa5e41b70eb4dcd08 is not leader of this config: current role FOLLOWER
W20260812 06:36:21.160120  7250 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:58454: Illegal state: replica b037d17b21f8422fa5e41b70eb4dcd08 is not leader of this config: current role FOLLOWER
W20260812 06:36:21.160596  7249 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:58454: Illegal state: replica b037d17b21f8422fa5e41b70eb4dcd08 is not leader of this config: current role FOLLOWER
W20260812 06:36:21.167040  7250 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:58454: Illegal state: replica b037d17b21f8422fa5e41b70eb4dcd08 is not leader of this config: current role FOLLOWER
W20260812 06:36:21.167616  7254 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:58454: Illegal state: replica b037d17b21f8422fa5e41b70eb4dcd08 is not leader of this config: current role FOLLOWER
W20260812 06:36:21.168109  7249 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:58454: Illegal state: replica b037d17b21f8422fa5e41b70eb4dcd08 is not leader of this config: current role FOLLOWER
W20260812 06:36:21.170744  7051 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50492: Illegal state: replica 2df4308c852240859ada53d216ab69ae is not leader of this config: current role FOLLOWER
W20260812 06:36:21.171926  7051 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50492: Illegal state: replica 2df4308c852240859ada53d216ab69ae is not leader of this config: current role FOLLOWER
W20260812 06:36:21.185380  5988 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:42334: Illegal state: replica ec6c8d818e744cdbb6b0d15b321010e0 is not leader of this config: current role FOLLOWER
W20260812 06:36:21.186082  5985 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:42334: Illegal state: replica ec6c8d818e744cdbb6b0d15b321010e0 is not leader of this config: current role FOLLOWER
W20260812 06:36:21.186538  5984 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:42334: Illegal state: replica ec6c8d818e744cdbb6b0d15b321010e0 is not leader of this config: current role FOLLOWER
W20260812 06:36:21.190876  5984 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:42334: Illegal state: replica ec6c8d818e744cdbb6b0d15b321010e0 is not leader of this config: current role FOLLOWER
W20260812 06:36:21.191823  5985 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:42334: Illegal state: replica ec6c8d818e744cdbb6b0d15b321010e0 is not leader of this config: current role FOLLOWER
I20260812 06:36:22.908659  7925 raft_consensus.cc:493] T 7065bb403e3047dd9a0853c49318ef1f P ec6c8d818e744cdbb6b0d15b321010e0 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 42895a9bcba64aa484f6509605035d02)
I20260812 06:36:22.908846  7925 raft_consensus.cc:515] T 7065bb403e3047dd9a0853c49318ef1f P ec6c8d818e744cdbb6b0d15b321010e0 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df4308c852240859ada53d216ab69ae" member_type: VOTER last_known_addr { host: "127.4.177.6" port: 34269 } } peers { permanent_uuid: "42895a9bcba64aa484f6509605035d02" member_type: VOTER last_known_addr { host: "127.4.177.4" port: 38181 } } peers { permanent_uuid: "222b20de5a1145c6b8a11df8374e8790" member_type: VOTER last_known_addr { host: "127.4.177.5" port: 43521 } } peers { permanent_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 40897 } } peers { permanent_uuid: "8d90d61da62549208fe117b4384a469b" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 41539 } } peers { permanent_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" member_type: VOTER last_known_addr { host: "127.4.177.7" port: 46299 } } peers { permanent_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 41057 } }
I20260812 06:36:22.910315  7108 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" candidate_term: 6 candidate_status { last_received { term: 5 index: 630 } } ignore_live_leader: false dest_uuid: "2df4308c852240859ada53d216ab69ae" is_pre_election: true
I20260812 06:36:22.910357  6247 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" candidate_term: 6 candidate_status { last_received { term: 5 index: 630 } } ignore_live_leader: false dest_uuid: "8d90d61da62549208fe117b4384a469b" is_pre_election: true
I20260812 06:36:22.910367  6919 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" candidate_term: 6 candidate_status { last_received { term: 5 index: 630 } } ignore_live_leader: false dest_uuid: "222b20de5a1145c6b8a11df8374e8790" is_pre_election: true
I20260812 06:36:22.910480  7321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" candidate_term: 6 candidate_status { last_received { term: 5 index: 630 } } ignore_live_leader: false dest_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" is_pre_election: true
I20260812 06:36:22.910725  6480 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" candidate_term: 6 candidate_status { last_received { term: 5 index: 630 } } ignore_live_leader: false dest_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" is_pre_election: true
I20260812 06:36:22.909881  7925 leader_election.cc:290] T 7065bb403e3047dd9a0853c49318ef1f P ec6c8d818e744cdbb6b0d15b321010e0 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 2df4308c852240859ada53d216ab69ae (127.4.177.6:34269), 42895a9bcba64aa484f6509605035d02 (127.4.177.4:38181), 222b20de5a1145c6b8a11df8374e8790 (127.4.177.5:43521), 7962f587ccd842ad8a3f6ed9a5d7707e (127.4.177.3:40897), 8d90d61da62549208fe117b4384a469b (127.4.177.2:41539), b037d17b21f8422fa5e41b70eb4dcd08 (127.4.177.7:46299)
I20260812 06:36:22.911048  5945 leader_election.cc:304] T 7065bb403e3047dd9a0853c49318ef1f P ec6c8d818e744cdbb6b0d15b321010e0 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 5 responses out of 7 voters: 1 yes votes; 4 no votes. yes voters: ec6c8d818e744cdbb6b0d15b321010e0; no voters: 222b20de5a1145c6b8a11df8374e8790, 2df4308c852240859ada53d216ab69ae, 8d90d61da62549208fe117b4384a469b, b037d17b21f8422fa5e41b70eb4dcd08
I20260812 06:36:22.911234  7925 raft_consensus.cc:2749] T 7065bb403e3047dd9a0853c49318ef1f P ec6c8d818e744cdbb6b0d15b321010e0 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260812 06:36:23.020382  7926 raft_consensus.cc:493] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 42895a9bcba64aa484f6509605035d02)
I20260812 06:36:23.020555  7926 raft_consensus.cc:515] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df4308c852240859ada53d216ab69ae" member_type: VOTER last_known_addr { host: "127.4.177.6" port: 34269 } } peers { permanent_uuid: "42895a9bcba64aa484f6509605035d02" member_type: VOTER last_known_addr { host: "127.4.177.4" port: 38181 } } peers { permanent_uuid: "222b20de5a1145c6b8a11df8374e8790" member_type: VOTER last_known_addr { host: "127.4.177.5" port: 43521 } } peers { permanent_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 40897 } } peers { permanent_uuid: "8d90d61da62549208fe117b4384a469b" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 41539 } } peers { permanent_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" member_type: VOTER last_known_addr { host: "127.4.177.7" port: 46299 } } peers { permanent_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 41057 } }
I20260812 06:36:23.021602  7108 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" candidate_term: 6 candidate_status { last_received { term: 5 index: 665 } } ignore_live_leader: false dest_uuid: "2df4308c852240859ada53d216ab69ae" is_pre_election: true
I20260812 06:36:23.021442  7926 leader_election.cc:290] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 2df4308c852240859ada53d216ab69ae (127.4.177.6:34269), 42895a9bcba64aa484f6509605035d02 (127.4.177.4:38181), 222b20de5a1145c6b8a11df8374e8790 (127.4.177.5:43521), 7962f587ccd842ad8a3f6ed9a5d7707e (127.4.177.3:40897), 8d90d61da62549208fe117b4384a469b (127.4.177.2:41539), ec6c8d818e744cdbb6b0d15b321010e0 (127.4.177.1:41057)
I20260812 06:36:23.022127  6247 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" candidate_term: 6 candidate_status { last_received { term: 5 index: 665 } } ignore_live_leader: false dest_uuid: "8d90d61da62549208fe117b4384a469b" is_pre_election: true
I20260812 06:36:23.022127  6919 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" candidate_term: 6 candidate_status { last_received { term: 5 index: 665 } } ignore_live_leader: false dest_uuid: "222b20de5a1145c6b8a11df8374e8790" is_pre_election: true
I20260812 06:36:23.022121  6480 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" candidate_term: 6 candidate_status { last_received { term: 5 index: 665 } } ignore_live_leader: false dest_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" is_pre_election: true
I20260812 06:36:23.022143  6047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" candidate_term: 6 candidate_status { last_received { term: 5 index: 665 } } ignore_live_leader: false dest_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" is_pre_election: true
I20260812 06:36:23.022303  6047 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P ec6c8d818e744cdbb6b0d15b321010e0 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b037d17b21f8422fa5e41b70eb4dcd08 in term 5.
I20260812 06:36:23.022929  7211 leader_election.cc:304] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 6 responses out of 7 voters: 2 yes votes; 4 no votes. yes voters: b037d17b21f8422fa5e41b70eb4dcd08, ec6c8d818e744cdbb6b0d15b321010e0; no voters: 222b20de5a1145c6b8a11df8374e8790, 2df4308c852240859ada53d216ab69ae, 7962f587ccd842ad8a3f6ed9a5d7707e, 8d90d61da62549208fe117b4384a469b
I20260812 06:36:23.023141  7926 raft_consensus.cc:2749] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260812 06:36:23.250509  7927 raft_consensus.cc:493] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 42895a9bcba64aa484f6509605035d02)
I20260812 06:36:23.250675  7927 raft_consensus.cc:515] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df4308c852240859ada53d216ab69ae" member_type: VOTER last_known_addr { host: "127.4.177.6" port: 34269 } } peers { permanent_uuid: "42895a9bcba64aa484f6509605035d02" member_type: VOTER last_known_addr { host: "127.4.177.4" port: 38181 } } peers { permanent_uuid: "222b20de5a1145c6b8a11df8374e8790" member_type: VOTER last_known_addr { host: "127.4.177.5" port: 43521 } } peers { permanent_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 40897 } } peers { permanent_uuid: "8d90d61da62549208fe117b4384a469b" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 41539 } } peers { permanent_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" member_type: VOTER last_known_addr { host: "127.4.177.7" port: 46299 } } peers { permanent_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 41057 } }
I20260812 06:36:23.251650  7927 leader_election.cc:290] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 2df4308c852240859ada53d216ab69ae (127.4.177.6:34269), 42895a9bcba64aa484f6509605035d02 (127.4.177.4:38181), 7962f587ccd842ad8a3f6ed9a5d7707e (127.4.177.3:40897), 8d90d61da62549208fe117b4384a469b (127.4.177.2:41539), b037d17b21f8422fa5e41b70eb4dcd08 (127.4.177.7:46299), ec6c8d818e744cdbb6b0d15b321010e0 (127.4.177.1:41057)
I20260812 06:36:23.252025  7108 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "222b20de5a1145c6b8a11df8374e8790" candidate_term: 6 candidate_status { last_received { term: 5 index: 757 } } ignore_live_leader: false dest_uuid: "2df4308c852240859ada53d216ab69ae" is_pre_election: true
I20260812 06:36:23.252140  7928 raft_consensus.cc:493] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 42895a9bcba64aa484f6509605035d02)
I20260812 06:36:23.252243  6480 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "222b20de5a1145c6b8a11df8374e8790" candidate_term: 6 candidate_status { last_received { term: 5 index: 757 } } ignore_live_leader: false dest_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" is_pre_election: true
I20260812 06:36:23.252257  7928 raft_consensus.cc:515] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df4308c852240859ada53d216ab69ae" member_type: VOTER last_known_addr { host: "127.4.177.6" port: 34269 } } peers { permanent_uuid: "42895a9bcba64aa484f6509605035d02" member_type: VOTER last_known_addr { host: "127.4.177.4" port: 38181 } } peers { permanent_uuid: "222b20de5a1145c6b8a11df8374e8790" member_type: VOTER last_known_addr { host: "127.4.177.5" port: 43521 } } peers { permanent_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 40897 } } peers { permanent_uuid: "8d90d61da62549208fe117b4384a469b" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 41539 } } peers { permanent_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" member_type: VOTER last_known_addr { host: "127.4.177.7" port: 46299 } } peers { permanent_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 41057 } }
I20260812 06:36:23.252489  6047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "222b20de5a1145c6b8a11df8374e8790" candidate_term: 6 candidate_status { last_received { term: 5 index: 757 } } ignore_live_leader: false dest_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" is_pre_election: true
I20260812 06:36:23.252627  6047 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P ec6c8d818e744cdbb6b0d15b321010e0 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 222b20de5a1145c6b8a11df8374e8790 in term 5.
I20260812 06:36:23.253185  7928 leader_election.cc:290] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 42895a9bcba64aa484f6509605035d02 (127.4.177.4:38181), 222b20de5a1145c6b8a11df8374e8790 (127.4.177.5:43521), 7962f587ccd842ad8a3f6ed9a5d7707e (127.4.177.3:40897), 8d90d61da62549208fe117b4384a469b (127.4.177.2:41539), b037d17b21f8422fa5e41b70eb4dcd08 (127.4.177.7:46299), ec6c8d818e744cdbb6b0d15b321010e0 (127.4.177.1:41057)
I20260812 06:36:23.253620  6047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "2df4308c852240859ada53d216ab69ae" candidate_term: 6 candidate_status { last_received { term: 5 index: 757 } } ignore_live_leader: false dest_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" is_pre_election: true
I20260812 06:36:23.253661  6247 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "2df4308c852240859ada53d216ab69ae" candidate_term: 6 candidate_status { last_received { term: 5 index: 757 } } ignore_live_leader: false dest_uuid: "8d90d61da62549208fe117b4384a469b" is_pre_election: true
I20260812 06:36:23.253752  6047 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P ec6c8d818e744cdbb6b0d15b321010e0 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2df4308c852240859ada53d216ab69ae in term 5.
I20260812 06:36:23.254170  6480 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "2df4308c852240859ada53d216ab69ae" candidate_term: 6 candidate_status { last_received { term: 5 index: 757 } } ignore_live_leader: false dest_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" is_pre_election: true
I20260812 06:36:23.254475  7320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "2df4308c852240859ada53d216ab69ae" candidate_term: 6 candidate_status { last_received { term: 5 index: 757 } } ignore_live_leader: false dest_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" is_pre_election: true
I20260812 06:36:23.254617  7320 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2df4308c852240859ada53d216ab69ae in term 5.
I20260812 06:36:23.254602  6247 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "222b20de5a1145c6b8a11df8374e8790" candidate_term: 6 candidate_status { last_received { term: 5 index: 757 } } ignore_live_leader: false dest_uuid: "8d90d61da62549208fe117b4384a469b" is_pre_election: true
I20260812 06:36:23.254861  6919 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "2df4308c852240859ada53d216ab69ae" candidate_term: 6 candidate_status { last_received { term: 5 index: 757 } } ignore_live_leader: false dest_uuid: "222b20de5a1145c6b8a11df8374e8790" is_pre_election: true
I20260812 06:36:23.254984  6919 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2df4308c852240859ada53d216ab69ae in term 5.
I20260812 06:36:23.255187  7321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "222b20de5a1145c6b8a11df8374e8790" candidate_term: 6 candidate_status { last_received { term: 5 index: 757 } } ignore_live_leader: false dest_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" is_pre_election: true
I20260812 06:36:23.255287  7321 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 222b20de5a1145c6b8a11df8374e8790 in term 5.
I20260812 06:36:23.255486  7108 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 222b20de5a1145c6b8a11df8374e8790 in term 5.
I20260812 06:36:23.255548  7020 leader_election.cc:304] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 6 responses out of 7 voters: 4 yes votes; 2 no votes. yes voters: 222b20de5a1145c6b8a11df8374e8790, 2df4308c852240859ada53d216ab69ae, b037d17b21f8422fa5e41b70eb4dcd08, ec6c8d818e744cdbb6b0d15b321010e0; no voters: 7962f587ccd842ad8a3f6ed9a5d7707e, 8d90d61da62549208fe117b4384a469b
I20260812 06:36:23.255945  6821 leader_election.cc:304] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 6 responses out of 7 voters: 4 yes votes; 2 no votes. yes voters: 222b20de5a1145c6b8a11df8374e8790, 2df4308c852240859ada53d216ab69ae, b037d17b21f8422fa5e41b70eb4dcd08, ec6c8d818e744cdbb6b0d15b321010e0; no voters: 7962f587ccd842ad8a3f6ed9a5d7707e, 8d90d61da62549208fe117b4384a469b
I20260812 06:36:23.255755  7928 raft_consensus.cc:2804] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260812 06:36:23.256179  7928 raft_consensus.cc:493] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [term 5 FOLLOWER]: Starting leader election (detected failure of leader 42895a9bcba64aa484f6509605035d02)
I20260812 06:36:23.256230  7928 raft_consensus.cc:3060] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [term 5 FOLLOWER]: Advancing to term 6
I20260812 06:36:23.256251  7927 raft_consensus.cc:2804] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260812 06:36:23.256385  7927 raft_consensus.cc:493] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 42895a9bcba64aa484f6509605035d02)
I20260812 06:36:23.256471  7927 raft_consensus.cc:3060] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [term 5 FOLLOWER]: Advancing to term 6
I20260812 06:36:23.257395  7928 raft_consensus.cc:515] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df4308c852240859ada53d216ab69ae" member_type: VOTER last_known_addr { host: "127.4.177.6" port: 34269 } } peers { permanent_uuid: "42895a9bcba64aa484f6509605035d02" member_type: VOTER last_known_addr { host: "127.4.177.4" port: 38181 } } peers { permanent_uuid: "222b20de5a1145c6b8a11df8374e8790" member_type: VOTER last_known_addr { host: "127.4.177.5" port: 43521 } } peers { permanent_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 40897 } } peers { permanent_uuid: "8d90d61da62549208fe117b4384a469b" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 41539 } } peers { permanent_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" member_type: VOTER last_known_addr { host: "127.4.177.7" port: 46299 } } peers { permanent_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 41057 } }
I20260812 06:36:23.257714  7927 raft_consensus.cc:515] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df4308c852240859ada53d216ab69ae" member_type: VOTER last_known_addr { host: "127.4.177.6" port: 34269 } } peers { permanent_uuid: "42895a9bcba64aa484f6509605035d02" member_type: VOTER last_known_addr { host: "127.4.177.4" port: 38181 } } peers { permanent_uuid: "222b20de5a1145c6b8a11df8374e8790" member_type: VOTER last_known_addr { host: "127.4.177.5" port: 43521 } } peers { permanent_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 40897 } } peers { permanent_uuid: "8d90d61da62549208fe117b4384a469b" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 41539 } } peers { permanent_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" member_type: VOTER last_known_addr { host: "127.4.177.7" port: 46299 } } peers { permanent_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 41057 } }
I20260812 06:36:23.258446  6919 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "2df4308c852240859ada53d216ab69ae" candidate_term: 6 candidate_status { last_received { term: 5 index: 757 } } ignore_live_leader: false dest_uuid: "222b20de5a1145c6b8a11df8374e8790"
I20260812 06:36:23.258603  6919 raft_consensus.cc:2393] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 2df4308c852240859ada53d216ab69ae in current term 6: Already voted for candidate 222b20de5a1145c6b8a11df8374e8790 in this term.
I20260812 06:36:23.258863  6480 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "222b20de5a1145c6b8a11df8374e8790" candidate_term: 6 candidate_status { last_received { term: 5 index: 757 } } ignore_live_leader: false dest_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e"
I20260812 06:36:23.259238  7321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "222b20de5a1145c6b8a11df8374e8790" candidate_term: 6 candidate_status { last_received { term: 5 index: 757 } } ignore_live_leader: false dest_uuid: "b037d17b21f8422fa5e41b70eb4dcd08"
I20260812 06:36:23.259377  7321 raft_consensus.cc:3060] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08 [term 5 FOLLOWER]: Advancing to term 6
I20260812 06:36:23.259672  6047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "222b20de5a1145c6b8a11df8374e8790" candidate_term: 6 candidate_status { last_received { term: 5 index: 757 } } ignore_live_leader: false dest_uuid: "ec6c8d818e744cdbb6b0d15b321010e0"
I20260812 06:36:23.259738  6247 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "222b20de5a1145c6b8a11df8374e8790" candidate_term: 6 candidate_status { last_received { term: 5 index: 757 } } ignore_live_leader: false dest_uuid: "8d90d61da62549208fe117b4384a469b"
I20260812 06:36:23.259784  6047 raft_consensus.cc:3060] T 7065bb403e3047dd9a0853c49318ef1f P ec6c8d818e744cdbb6b0d15b321010e0 [term 5 FOLLOWER]: Advancing to term 6
I20260812 06:36:23.260649  7321 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 222b20de5a1145c6b8a11df8374e8790 in term 6.
I20260812 06:36:23.260751  6480 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "2df4308c852240859ada53d216ab69ae" candidate_term: 6 candidate_status { last_received { term: 5 index: 757 } } ignore_live_leader: false dest_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e"
I20260812 06:36:23.261013  6047 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P ec6c8d818e744cdbb6b0d15b321010e0 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 222b20de5a1145c6b8a11df8374e8790 in term 6.
I20260812 06:36:23.261010  7108 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "222b20de5a1145c6b8a11df8374e8790" candidate_term: 6 candidate_status { last_received { term: 5 index: 757 } } ignore_live_leader: false dest_uuid: "2df4308c852240859ada53d216ab69ae"
I20260812 06:36:23.261157  7108 raft_consensus.cc:2393] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 222b20de5a1145c6b8a11df8374e8790 in current term 6: Already voted for candidate 2df4308c852240859ada53d216ab69ae in this term.
I20260812 06:36:23.261631  7928 leader_election.cc:290] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [CANDIDATE]: Term 6 election: Requested vote from peers 42895a9bcba64aa484f6509605035d02 (127.4.177.4:38181), 222b20de5a1145c6b8a11df8374e8790 (127.4.177.5:43521), 7962f587ccd842ad8a3f6ed9a5d7707e (127.4.177.3:40897), 8d90d61da62549208fe117b4384a469b (127.4.177.2:41539), b037d17b21f8422fa5e41b70eb4dcd08 (127.4.177.7:46299), ec6c8d818e744cdbb6b0d15b321010e0 (127.4.177.1:41057)
I20260812 06:36:23.261262  7927 leader_election.cc:290] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [CANDIDATE]: Term 6 election: Requested vote from peers 2df4308c852240859ada53d216ab69ae (127.4.177.6:34269), 42895a9bcba64aa484f6509605035d02 (127.4.177.4:38181), 7962f587ccd842ad8a3f6ed9a5d7707e (127.4.177.3:40897), 8d90d61da62549208fe117b4384a469b (127.4.177.2:41539), b037d17b21f8422fa5e41b70eb4dcd08 (127.4.177.7:46299), ec6c8d818e744cdbb6b0d15b321010e0 (127.4.177.1:41057)
I20260812 06:36:23.261965  6247 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "2df4308c852240859ada53d216ab69ae" candidate_term: 6 candidate_status { last_received { term: 5 index: 757 } } ignore_live_leader: false dest_uuid: "8d90d61da62549208fe117b4384a469b"
I20260812 06:36:23.262002  7321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "2df4308c852240859ada53d216ab69ae" candidate_term: 6 candidate_status { last_received { term: 5 index: 757 } } ignore_live_leader: false dest_uuid: "b037d17b21f8422fa5e41b70eb4dcd08"
I20260812 06:36:23.262179  7321 raft_consensus.cc:2393] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 2df4308c852240859ada53d216ab69ae in current term 6: Already voted for candidate 222b20de5a1145c6b8a11df8374e8790 in this term.
I20260812 06:36:23.262154  6047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "2df4308c852240859ada53d216ab69ae" candidate_term: 6 candidate_status { last_received { term: 5 index: 757 } } ignore_live_leader: false dest_uuid: "ec6c8d818e744cdbb6b0d15b321010e0"
I20260812 06:36:23.262332  6047 raft_consensus.cc:2393] T 7065bb403e3047dd9a0853c49318ef1f P ec6c8d818e744cdbb6b0d15b321010e0 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 2df4308c852240859ada53d216ab69ae in current term 6: Already voted for candidate 222b20de5a1145c6b8a11df8374e8790 in this term.
I20260812 06:36:23.262689  7020 leader_election.cc:304] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 5 responses out of 7 voters: 1 yes votes; 4 no votes. yes voters: 2df4308c852240859ada53d216ab69ae; no voters: 222b20de5a1145c6b8a11df8374e8790, 7962f587ccd842ad8a3f6ed9a5d7707e, 8d90d61da62549208fe117b4384a469b, ec6c8d818e744cdbb6b0d15b321010e0
I20260812 06:36:23.262903  7928 raft_consensus.cc:2749] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260812 06:36:23.590935  7930 raft_consensus.cc:493] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 42895a9bcba64aa484f6509605035d02)
I20260812 06:36:23.591081  7930 raft_consensus.cc:515] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df4308c852240859ada53d216ab69ae" member_type: VOTER last_known_addr { host: "127.4.177.6" port: 34269 } } peers { permanent_uuid: "42895a9bcba64aa484f6509605035d02" member_type: VOTER last_known_addr { host: "127.4.177.4" port: 38181 } } peers { permanent_uuid: "222b20de5a1145c6b8a11df8374e8790" member_type: VOTER last_known_addr { host: "127.4.177.5" port: 43521 } } peers { permanent_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 40897 } } peers { permanent_uuid: "8d90d61da62549208fe117b4384a469b" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 41539 } } peers { permanent_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" member_type: VOTER last_known_addr { host: "127.4.177.7" port: 46299 } } peers { permanent_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 41057 } }
I20260812 06:36:23.591923  7930 leader_election.cc:290] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 2df4308c852240859ada53d216ab69ae (127.4.177.6:34269), 42895a9bcba64aa484f6509605035d02 (127.4.177.4:38181), 222b20de5a1145c6b8a11df8374e8790 (127.4.177.5:43521), 7962f587ccd842ad8a3f6ed9a5d7707e (127.4.177.3:40897), b037d17b21f8422fa5e41b70eb4dcd08 (127.4.177.7:46299), ec6c8d818e744cdbb6b0d15b321010e0 (127.4.177.1:41057)
I20260812 06:36:23.592226  7108 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "8d90d61da62549208fe117b4384a469b" candidate_term: 6 candidate_status { last_received { term: 5 index: 710 } } ignore_live_leader: false dest_uuid: "2df4308c852240859ada53d216ab69ae" is_pre_election: true
I20260812 06:36:23.592412  7108 raft_consensus.cc:2393] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8d90d61da62549208fe117b4384a469b in current term 6: Already voted for candidate 2df4308c852240859ada53d216ab69ae in this term.
I20260812 06:36:23.592389  7321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "8d90d61da62549208fe117b4384a469b" candidate_term: 6 candidate_status { last_received { term: 5 index: 710 } } ignore_live_leader: false dest_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" is_pre_election: true
I20260812 06:36:23.592572  7321 raft_consensus.cc:2393] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8d90d61da62549208fe117b4384a469b in current term 6: Already voted for candidate 222b20de5a1145c6b8a11df8374e8790 in this term.
I20260812 06:36:23.592665  6919 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "8d90d61da62549208fe117b4384a469b" candidate_term: 6 candidate_status { last_received { term: 5 index: 710 } } ignore_live_leader: false dest_uuid: "222b20de5a1145c6b8a11df8374e8790" is_pre_election: true
I20260812 06:36:23.592832  6919 raft_consensus.cc:2393] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8d90d61da62549208fe117b4384a469b in current term 6: Already voted for candidate 222b20de5a1145c6b8a11df8374e8790 in this term.
I20260812 06:36:23.592811  6480 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "8d90d61da62549208fe117b4384a469b" candidate_term: 6 candidate_status { last_received { term: 5 index: 710 } } ignore_live_leader: false dest_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" is_pre_election: true
I20260812 06:36:23.593014  6480 raft_consensus.cc:2410] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8d90d61da62549208fe117b4384a469b for term 6 because replica has last-logged OpId of term: 5 index: 753, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 710.
I20260812 06:36:23.592989  6047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "8d90d61da62549208fe117b4384a469b" candidate_term: 6 candidate_status { last_received { term: 5 index: 710 } } ignore_live_leader: false dest_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" is_pre_election: true
I20260812 06:36:23.593148  6047 raft_consensus.cc:2393] T 7065bb403e3047dd9a0853c49318ef1f P ec6c8d818e744cdbb6b0d15b321010e0 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8d90d61da62549208fe117b4384a469b in current term 6: Already voted for candidate 222b20de5a1145c6b8a11df8374e8790 in this term.
I20260812 06:36:23.593179  7931 raft_consensus.cc:493] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 42895a9bcba64aa484f6509605035d02)
I20260812 06:36:23.593264  7931 raft_consensus.cc:515] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df4308c852240859ada53d216ab69ae" member_type: VOTER last_known_addr { host: "127.4.177.6" port: 34269 } } peers { permanent_uuid: "42895a9bcba64aa484f6509605035d02" member_type: VOTER last_known_addr { host: "127.4.177.4" port: 38181 } } peers { permanent_uuid: "222b20de5a1145c6b8a11df8374e8790" member_type: VOTER last_known_addr { host: "127.4.177.5" port: 43521 } } peers { permanent_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 40897 } } peers { permanent_uuid: "8d90d61da62549208fe117b4384a469b" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 41539 } } peers { permanent_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" member_type: VOTER last_known_addr { host: "127.4.177.7" port: 46299 } } peers { permanent_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 41057 } }
I20260812 06:36:23.593312  6150 leader_election.cc:304] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 5 responses out of 7 voters: 1 yes votes; 4 no votes. yes voters: 8d90d61da62549208fe117b4384a469b; no voters: 222b20de5a1145c6b8a11df8374e8790, 2df4308c852240859ada53d216ab69ae, 7962f587ccd842ad8a3f6ed9a5d7707e, b037d17b21f8422fa5e41b70eb4dcd08
I20260812 06:36:23.593487  7930 raft_consensus.cc:3060] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b [term 5 FOLLOWER]: Advancing to term 6
I20260812 06:36:23.594080  7108 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" candidate_term: 6 candidate_status { last_received { term: 5 index: 753 } } ignore_live_leader: false dest_uuid: "2df4308c852240859ada53d216ab69ae" is_pre_election: true
I20260812 06:36:23.594296  7108 raft_consensus.cc:2393] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7962f587ccd842ad8a3f6ed9a5d7707e in current term 6: Already voted for candidate 2df4308c852240859ada53d216ab69ae in this term.
I20260812 06:36:23.594285  6247 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" candidate_term: 6 candidate_status { last_received { term: 5 index: 753 } } ignore_live_leader: false dest_uuid: "8d90d61da62549208fe117b4384a469b" is_pre_election: true
I20260812 06:36:23.594398  6919 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" candidate_term: 6 candidate_status { last_received { term: 5 index: 753 } } ignore_live_leader: false dest_uuid: "222b20de5a1145c6b8a11df8374e8790" is_pre_election: true
I20260812 06:36:23.594503  7931 leader_election.cc:290] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 2df4308c852240859ada53d216ab69ae (127.4.177.6:34269), 42895a9bcba64aa484f6509605035d02 (127.4.177.4:38181), 222b20de5a1145c6b8a11df8374e8790 (127.4.177.5:43521), 8d90d61da62549208fe117b4384a469b (127.4.177.2:41539), b037d17b21f8422fa5e41b70eb4dcd08 (127.4.177.7:46299), ec6c8d818e744cdbb6b0d15b321010e0 (127.4.177.1:41057)
I20260812 06:36:23.594527  6919 raft_consensus.cc:2393] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7962f587ccd842ad8a3f6ed9a5d7707e in current term 6: Already voted for candidate 222b20de5a1145c6b8a11df8374e8790 in this term.
I20260812 06:36:23.594972  7321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" candidate_term: 6 candidate_status { last_received { term: 5 index: 753 } } ignore_live_leader: false dest_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" is_pre_election: true
I20260812 06:36:23.595032  6047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" candidate_term: 6 candidate_status { last_received { term: 5 index: 753 } } ignore_live_leader: false dest_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" is_pre_election: true
I20260812 06:36:23.595124  7321 raft_consensus.cc:2393] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7962f587ccd842ad8a3f6ed9a5d7707e in current term 6: Already voted for candidate 222b20de5a1145c6b8a11df8374e8790 in this term.
I20260812 06:36:23.595151  6047 raft_consensus.cc:2393] T 7065bb403e3047dd9a0853c49318ef1f P ec6c8d818e744cdbb6b0d15b321010e0 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7962f587ccd842ad8a3f6ed9a5d7707e in current term 6: Already voted for candidate 222b20de5a1145c6b8a11df8374e8790 in this term.
I20260812 06:36:23.595144  7930 raft_consensus.cc:2749] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260812 06:36:23.595341  6247 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7962f587ccd842ad8a3f6ed9a5d7707e in term 6.
I20260812 06:36:23.595482  6369 leader_election.cc:304] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 5 responses out of 7 voters: 1 yes votes; 4 no votes. yes voters: 7962f587ccd842ad8a3f6ed9a5d7707e; no voters: 222b20de5a1145c6b8a11df8374e8790, 2df4308c852240859ada53d216ab69ae, b037d17b21f8422fa5e41b70eb4dcd08, ec6c8d818e744cdbb6b0d15b321010e0
I20260812 06:36:23.595675  7931 raft_consensus.cc:3060] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [term 5 FOLLOWER]: Advancing to term 6
I20260812 06:36:23.596521  7931 raft_consensus.cc:2749] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260812 06:36:24.918500  7211 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.4.177.4:38181 timed out after 1.895s (SENT)
W20260812 06:36:24.957000  6821 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.4.177.4:38181 timed out after 1.704s (SENT)
W20260812 06:36:25.092793  7020 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.4.177.4:38181 timed out after 1.838s (SENT) [suppressed 1 similar messages]
W20260812 06:36:25.128720  5945 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.4.177.4:38181 timed out after 2.217s (SENT)
I20260812 06:36:25.182839  7940 raft_consensus.cc:493] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 06:36:25.183030  7940 raft_consensus.cc:515] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df4308c852240859ada53d216ab69ae" member_type: VOTER last_known_addr { host: "127.4.177.6" port: 34269 } } peers { permanent_uuid: "42895a9bcba64aa484f6509605035d02" member_type: VOTER last_known_addr { host: "127.4.177.4" port: 38181 } } peers { permanent_uuid: "222b20de5a1145c6b8a11df8374e8790" member_type: VOTER last_known_addr { host: "127.4.177.5" port: 43521 } } peers { permanent_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 40897 } } peers { permanent_uuid: "8d90d61da62549208fe117b4384a469b" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 41539 } } peers { permanent_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" member_type: VOTER last_known_addr { host: "127.4.177.7" port: 46299 } } peers { permanent_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 41057 } }
I20260812 06:36:25.183682  7940 leader_election.cc:290] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 2df4308c852240859ada53d216ab69ae (127.4.177.6:34269), 42895a9bcba64aa484f6509605035d02 (127.4.177.4:38181), 222b20de5a1145c6b8a11df8374e8790 (127.4.177.5:43521), 8d90d61da62549208fe117b4384a469b (127.4.177.2:41539), b037d17b21f8422fa5e41b70eb4dcd08 (127.4.177.7:46299), ec6c8d818e744cdbb6b0d15b321010e0 (127.4.177.1:41057)
I20260812 06:36:25.184085  7321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" candidate_term: 7 candidate_status { last_received { term: 5 index: 753 } } ignore_live_leader: false dest_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" is_pre_election: true
I20260812 06:36:25.184202  7108 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" candidate_term: 7 candidate_status { last_received { term: 5 index: 753 } } ignore_live_leader: false dest_uuid: "2df4308c852240859ada53d216ab69ae" is_pre_election: true
I20260812 06:36:25.184239  7321 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7962f587ccd842ad8a3f6ed9a5d7707e in term 6.
I20260812 06:36:25.184250  6247 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" candidate_term: 7 candidate_status { last_received { term: 5 index: 753 } } ignore_live_leader: false dest_uuid: "8d90d61da62549208fe117b4384a469b" is_pre_election: true
I20260812 06:36:25.184250  6047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" candidate_term: 7 candidate_status { last_received { term: 5 index: 753 } } ignore_live_leader: false dest_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" is_pre_election: true
I20260812 06:36:25.184353  7108 raft_consensus.cc:2410] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7962f587ccd842ad8a3f6ed9a5d7707e for term 7 because replica has last-logged OpId of term: 5 index: 757, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 753.
I20260812 06:36:25.184409  6247 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7962f587ccd842ad8a3f6ed9a5d7707e in term 6.
I20260812 06:36:25.184445  6047 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P ec6c8d818e744cdbb6b0d15b321010e0 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7962f587ccd842ad8a3f6ed9a5d7707e in term 6.
I20260812 06:36:25.184531  6919 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" candidate_term: 7 candidate_status { last_received { term: 5 index: 753 } } ignore_live_leader: false dest_uuid: "222b20de5a1145c6b8a11df8374e8790" is_pre_election: true
I20260812 06:36:25.184707  6919 raft_consensus.cc:2410] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7962f587ccd842ad8a3f6ed9a5d7707e for term 7 because replica has last-logged OpId of term: 5 index: 757, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 753.
I20260812 06:36:25.184840  6364 leader_election.cc:304] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 7 voters: 4 yes votes; 0 no votes. yes voters: 7962f587ccd842ad8a3f6ed9a5d7707e, 8d90d61da62549208fe117b4384a469b, b037d17b21f8422fa5e41b70eb4dcd08, ec6c8d818e744cdbb6b0d15b321010e0; no voters: 
I20260812 06:36:25.185091  7940 raft_consensus.cc:2804] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260812 06:36:25.185153  7940 raft_consensus.cc:493] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 06:36:25.185182  7940 raft_consensus.cc:3060] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [term 6 FOLLOWER]: Advancing to term 7
I20260812 06:36:25.186254  7940 raft_consensus.cc:515] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df4308c852240859ada53d216ab69ae" member_type: VOTER last_known_addr { host: "127.4.177.6" port: 34269 } } peers { permanent_uuid: "42895a9bcba64aa484f6509605035d02" member_type: VOTER last_known_addr { host: "127.4.177.4" port: 38181 } } peers { permanent_uuid: "222b20de5a1145c6b8a11df8374e8790" member_type: VOTER last_known_addr { host: "127.4.177.5" port: 43521 } } peers { permanent_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 40897 } } peers { permanent_uuid: "8d90d61da62549208fe117b4384a469b" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 41539 } } peers { permanent_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" member_type: VOTER last_known_addr { host: "127.4.177.7" port: 46299 } } peers { permanent_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 41057 } }
I20260812 06:36:25.186899  7940 leader_election.cc:290] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [CANDIDATE]: Term 7 election: Requested vote from peers 2df4308c852240859ada53d216ab69ae (127.4.177.6:34269), 42895a9bcba64aa484f6509605035d02 (127.4.177.4:38181), 222b20de5a1145c6b8a11df8374e8790 (127.4.177.5:43521), 8d90d61da62549208fe117b4384a469b (127.4.177.2:41539), b037d17b21f8422fa5e41b70eb4dcd08 (127.4.177.7:46299), ec6c8d818e744cdbb6b0d15b321010e0 (127.4.177.1:41057)
I20260812 06:36:25.187162  7108 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" candidate_term: 7 candidate_status { last_received { term: 5 index: 753 } } ignore_live_leader: false dest_uuid: "2df4308c852240859ada53d216ab69ae"
I20260812 06:36:25.187242  7321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" candidate_term: 7 candidate_status { last_received { term: 5 index: 753 } } ignore_live_leader: false dest_uuid: "b037d17b21f8422fa5e41b70eb4dcd08"
I20260812 06:36:25.187282  7108 raft_consensus.cc:3060] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [term 6 FOLLOWER]: Advancing to term 7
I20260812 06:36:25.187270  6247 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" candidate_term: 7 candidate_status { last_received { term: 5 index: 753 } } ignore_live_leader: false dest_uuid: "8d90d61da62549208fe117b4384a469b"
I20260812 06:36:25.187345  7321 raft_consensus.cc:3060] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08 [term 6 FOLLOWER]: Advancing to term 7
I20260812 06:36:25.187386  6247 raft_consensus.cc:3060] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b [term 6 FOLLOWER]: Advancing to term 7
I20260812 06:36:25.187508  6047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" candidate_term: 7 candidate_status { last_received { term: 5 index: 753 } } ignore_live_leader: false dest_uuid: "ec6c8d818e744cdbb6b0d15b321010e0"
I20260812 06:36:25.187623  6047 raft_consensus.cc:3060] T 7065bb403e3047dd9a0853c49318ef1f P ec6c8d818e744cdbb6b0d15b321010e0 [term 6 FOLLOWER]: Advancing to term 7
I20260812 06:36:25.188316  7321 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7962f587ccd842ad8a3f6ed9a5d7707e in term 7.
I20260812 06:36:25.188367  7108 raft_consensus.cc:2410] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 7962f587ccd842ad8a3f6ed9a5d7707e for term 7 because replica has last-logged OpId of term: 5 index: 757, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 753.
I20260812 06:36:25.188525  6247 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7962f587ccd842ad8a3f6ed9a5d7707e in term 7.
I20260812 06:36:25.188630  6047 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P ec6c8d818e744cdbb6b0d15b321010e0 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7962f587ccd842ad8a3f6ed9a5d7707e in term 7.
I20260812 06:36:25.188608  6919 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" candidate_term: 7 candidate_status { last_received { term: 5 index: 753 } } ignore_live_leader: false dest_uuid: "222b20de5a1145c6b8a11df8374e8790"
I20260812 06:36:25.188727  6919 raft_consensus.cc:3060] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [term 6 FOLLOWER]: Advancing to term 7
I20260812 06:36:25.189062  6369 leader_election.cc:304] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 5 responses out of 7 voters: 4 yes votes; 1 no votes. yes voters: 7962f587ccd842ad8a3f6ed9a5d7707e, 8d90d61da62549208fe117b4384a469b, b037d17b21f8422fa5e41b70eb4dcd08, ec6c8d818e744cdbb6b0d15b321010e0; no voters: 2df4308c852240859ada53d216ab69ae
I20260812 06:36:25.189241  7940 raft_consensus.cc:2804] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [term 7 FOLLOWER]: Leader election won for term 7
I20260812 06:36:25.189422  7940 raft_consensus.cc:697] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [term 7 LEADER]: Becoming Leader. State: Replica: 7962f587ccd842ad8a3f6ed9a5d7707e, State: Running, Role: LEADER
I20260812 06:36:25.189594  7940 consensus_queue.cc:237] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 747, Committed index: 747, Last appended: 5.753, Last appended by leader: 753, Current term: 7, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df4308c852240859ada53d216ab69ae" member_type: VOTER last_known_addr { host: "127.4.177.6" port: 34269 } } peers { permanent_uuid: "42895a9bcba64aa484f6509605035d02" member_type: VOTER last_known_addr { host: "127.4.177.4" port: 38181 } } peers { permanent_uuid: "222b20de5a1145c6b8a11df8374e8790" member_type: VOTER last_known_addr { host: "127.4.177.5" port: 43521 } } peers { permanent_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 40897 } } peers { permanent_uuid: "8d90d61da62549208fe117b4384a469b" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 41539 } } peers { permanent_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" member_type: VOTER last_known_addr { host: "127.4.177.7" port: 46299 } } peers { permanent_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 41057 } }
I20260812 06:36:25.189687  6919 raft_consensus.cc:2410] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 7962f587ccd842ad8a3f6ed9a5d7707e for term 7 because replica has last-logged OpId of term: 5 index: 757, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 753.
I20260812 06:36:25.192004  5812 catalog_manager.cc:5719] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e reported cstate change: term changed from 5 to 7, leader changed from 42895a9bcba64aa484f6509605035d02 (127.4.177.4) to 7962f587ccd842ad8a3f6ed9a5d7707e (127.4.177.3). New cstate: current_term: 7 leader_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df4308c852240859ada53d216ab69ae" member_type: VOTER last_known_addr { host: "127.4.177.6" port: 34269 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "42895a9bcba64aa484f6509605035d02" member_type: VOTER last_known_addr { host: "127.4.177.4" port: 38181 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "222b20de5a1145c6b8a11df8374e8790" member_type: VOTER last_known_addr { host: "127.4.177.5" port: 43521 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 40897 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8d90d61da62549208fe117b4384a469b" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 41539 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" member_type: VOTER last_known_addr { host: "127.4.177.7" port: 46299 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 41057 } health_report { overall_health: UNKNOWN } } }
I20260812 06:36:25.345291  7958 raft_consensus.cc:493] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 06:36:25.345439  7958 raft_consensus.cc:515] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df4308c852240859ada53d216ab69ae" member_type: VOTER last_known_addr { host: "127.4.177.6" port: 34269 } } peers { permanent_uuid: "42895a9bcba64aa484f6509605035d02" member_type: VOTER last_known_addr { host: "127.4.177.4" port: 38181 } } peers { permanent_uuid: "222b20de5a1145c6b8a11df8374e8790" member_type: VOTER last_known_addr { host: "127.4.177.5" port: 43521 } } peers { permanent_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 40897 } } peers { permanent_uuid: "8d90d61da62549208fe117b4384a469b" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 41539 } } peers { permanent_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" member_type: VOTER last_known_addr { host: "127.4.177.7" port: 46299 } } peers { permanent_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 41057 } }
I20260812 06:36:25.346112  7958 leader_election.cc:290] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 42895a9bcba64aa484f6509605035d02 (127.4.177.4:38181), 222b20de5a1145c6b8a11df8374e8790 (127.4.177.5:43521), 7962f587ccd842ad8a3f6ed9a5d7707e (127.4.177.3:40897), 8d90d61da62549208fe117b4384a469b (127.4.177.2:41539), b037d17b21f8422fa5e41b70eb4dcd08 (127.4.177.7:46299), ec6c8d818e744cdbb6b0d15b321010e0 (127.4.177.1:41057)
I20260812 06:36:25.346567  7321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "2df4308c852240859ada53d216ab69ae" candidate_term: 8 candidate_status { last_received { term: 5 index: 757 } } ignore_live_leader: false dest_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" is_pre_election: true
I20260812 06:36:25.346649  6480 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "2df4308c852240859ada53d216ab69ae" candidate_term: 8 candidate_status { last_received { term: 5 index: 757 } } ignore_live_leader: false dest_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" is_pre_election: true
I20260812 06:36:25.346633  6919 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "2df4308c852240859ada53d216ab69ae" candidate_term: 8 candidate_status { last_received { term: 5 index: 757 } } ignore_live_leader: false dest_uuid: "222b20de5a1145c6b8a11df8374e8790" is_pre_election: true
I20260812 06:36:25.346735  7321 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2df4308c852240859ada53d216ab69ae in term 7.
I20260812 06:36:25.346771  6919 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2df4308c852240859ada53d216ab69ae in term 7.
I20260812 06:36:25.346768  6047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "2df4308c852240859ada53d216ab69ae" candidate_term: 8 candidate_status { last_received { term: 5 index: 757 } } ignore_live_leader: false dest_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" is_pre_election: true
I20260812 06:36:25.346895  6047 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P ec6c8d818e744cdbb6b0d15b321010e0 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2df4308c852240859ada53d216ab69ae in term 7.
I20260812 06:36:25.347373  6247 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "2df4308c852240859ada53d216ab69ae" candidate_term: 8 candidate_status { last_received { term: 5 index: 757 } } ignore_live_leader: false dest_uuid: "8d90d61da62549208fe117b4384a469b" is_pre_election: true
I20260812 06:36:25.347421  7020 leader_election.cc:304] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 5 responses out of 7 voters: 4 yes votes; 1 no votes. yes voters: 222b20de5a1145c6b8a11df8374e8790, 2df4308c852240859ada53d216ab69ae, b037d17b21f8422fa5e41b70eb4dcd08, ec6c8d818e744cdbb6b0d15b321010e0; no voters: 7962f587ccd842ad8a3f6ed9a5d7707e
I20260812 06:36:25.347515  6247 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2df4308c852240859ada53d216ab69ae in term 7.
I20260812 06:36:25.347642  7958 raft_consensus.cc:2804] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260812 06:36:25.347719  7958 raft_consensus.cc:493] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 06:36:25.347774  7958 raft_consensus.cc:3060] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [term 7 FOLLOWER]: Advancing to term 8
I20260812 06:36:25.348671  7958 raft_consensus.cc:515] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df4308c852240859ada53d216ab69ae" member_type: VOTER last_known_addr { host: "127.4.177.6" port: 34269 } } peers { permanent_uuid: "42895a9bcba64aa484f6509605035d02" member_type: VOTER last_known_addr { host: "127.4.177.4" port: 38181 } } peers { permanent_uuid: "222b20de5a1145c6b8a11df8374e8790" member_type: VOTER last_known_addr { host: "127.4.177.5" port: 43521 } } peers { permanent_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 40897 } } peers { permanent_uuid: "8d90d61da62549208fe117b4384a469b" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 41539 } } peers { permanent_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" member_type: VOTER last_known_addr { host: "127.4.177.7" port: 46299 } } peers { permanent_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 41057 } }
I20260812 06:36:25.349170  7958 leader_election.cc:290] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [CANDIDATE]: Term 8 election: Requested vote from peers 42895a9bcba64aa484f6509605035d02 (127.4.177.4:38181), 222b20de5a1145c6b8a11df8374e8790 (127.4.177.5:43521), 7962f587ccd842ad8a3f6ed9a5d7707e (127.4.177.3:40897), 8d90d61da62549208fe117b4384a469b (127.4.177.2:41539), b037d17b21f8422fa5e41b70eb4dcd08 (127.4.177.7:46299), ec6c8d818e744cdbb6b0d15b321010e0 (127.4.177.1:41057)
I20260812 06:36:25.349592  7321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "2df4308c852240859ada53d216ab69ae" candidate_term: 8 candidate_status { last_received { term: 5 index: 757 } } ignore_live_leader: false dest_uuid: "b037d17b21f8422fa5e41b70eb4dcd08"
I20260812 06:36:25.349617  6480 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "2df4308c852240859ada53d216ab69ae" candidate_term: 8 candidate_status { last_received { term: 5 index: 757 } } ignore_live_leader: false dest_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e"
I20260812 06:36:25.349604  6919 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "2df4308c852240859ada53d216ab69ae" candidate_term: 8 candidate_status { last_received { term: 5 index: 757 } } ignore_live_leader: false dest_uuid: "222b20de5a1145c6b8a11df8374e8790"
I20260812 06:36:25.349730  6919 raft_consensus.cc:3060] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [term 7 FOLLOWER]: Advancing to term 8
I20260812 06:36:25.349731  7321 raft_consensus.cc:3060] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08 [term 7 FOLLOWER]: Advancing to term 8
I20260812 06:36:25.349968  6247 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "2df4308c852240859ada53d216ab69ae" candidate_term: 8 candidate_status { last_received { term: 5 index: 757 } } ignore_live_leader: false dest_uuid: "8d90d61da62549208fe117b4384a469b"
I20260812 06:36:25.350085  6247 raft_consensus.cc:3060] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b [term 7 FOLLOWER]: Advancing to term 8
I20260812 06:36:25.350075  6047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "2df4308c852240859ada53d216ab69ae" candidate_term: 8 candidate_status { last_received { term: 5 index: 757 } } ignore_live_leader: false dest_uuid: "ec6c8d818e744cdbb6b0d15b321010e0"
I20260812 06:36:25.350191  6047 raft_consensus.cc:3060] T 7065bb403e3047dd9a0853c49318ef1f P ec6c8d818e744cdbb6b0d15b321010e0 [term 7 FOLLOWER]: Advancing to term 8
I20260812 06:36:25.350896  6919 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2df4308c852240859ada53d216ab69ae in term 8.
I20260812 06:36:25.350912  7321 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2df4308c852240859ada53d216ab69ae in term 8.
I20260812 06:36:25.351157  6247 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2df4308c852240859ada53d216ab69ae in term 8.
I20260812 06:36:25.351423  6047 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P ec6c8d818e744cdbb6b0d15b321010e0 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2df4308c852240859ada53d216ab69ae in term 8.
I20260812 06:36:25.351445  7020 leader_election.cc:304] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 5 responses out of 7 voters: 4 yes votes; 1 no votes. yes voters: 222b20de5a1145c6b8a11df8374e8790, 2df4308c852240859ada53d216ab69ae, 8d90d61da62549208fe117b4384a469b, b037d17b21f8422fa5e41b70eb4dcd08; no voters: 7962f587ccd842ad8a3f6ed9a5d7707e
I20260812 06:36:25.351652  7958 raft_consensus.cc:2804] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [term 8 FOLLOWER]: Leader election won for term 8
I20260812 06:36:25.351939  7958 raft_consensus.cc:697] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [term 8 LEADER]: Becoming Leader. State: Replica: 2df4308c852240859ada53d216ab69ae, State: Running, Role: LEADER
I20260812 06:36:25.352317  7958 consensus_queue.cc:237] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 749, Committed index: 749, Last appended: 5.757, Last appended by leader: 757, Current term: 8, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df4308c852240859ada53d216ab69ae" member_type: VOTER last_known_addr { host: "127.4.177.6" port: 34269 } } peers { permanent_uuid: "42895a9bcba64aa484f6509605035d02" member_type: VOTER last_known_addr { host: "127.4.177.4" port: 38181 } } peers { permanent_uuid: "222b20de5a1145c6b8a11df8374e8790" member_type: VOTER last_known_addr { host: "127.4.177.5" port: 43521 } } peers { permanent_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 40897 } } peers { permanent_uuid: "8d90d61da62549208fe117b4384a469b" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 41539 } } peers { permanent_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" member_type: VOTER last_known_addr { host: "127.4.177.7" port: 46299 } } peers { permanent_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 41057 } }
I20260812 06:36:25.355080  5812 catalog_manager.cc:5719] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae reported cstate change: term changed from 7 to 8, leader changed from 7962f587ccd842ad8a3f6ed9a5d7707e (127.4.177.3) to 2df4308c852240859ada53d216ab69ae (127.4.177.6). New cstate: current_term: 8 leader_uuid: "2df4308c852240859ada53d216ab69ae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df4308c852240859ada53d216ab69ae" member_type: VOTER last_known_addr { host: "127.4.177.6" port: 34269 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "42895a9bcba64aa484f6509605035d02" member_type: VOTER last_known_addr { host: "127.4.177.4" port: 38181 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "222b20de5a1145c6b8a11df8374e8790" member_type: VOTER last_known_addr { host: "127.4.177.5" port: 43521 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 40897 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8d90d61da62549208fe117b4384a469b" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 41539 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" member_type: VOTER last_known_addr { host: "127.4.177.7" port: 46299 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 41057 } health_report { overall_health: UNKNOWN } } }
W20260812 06:36:25.381668  6150 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.4.177.4:38181 timed out after 1.789s (SENT)
W20260812 06:36:25.453526  6595 connection.cc:256] RPC call timeout handler was delayed by -3.0357685082126409s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1s for a response
I20260812 06:36:25.453984  6703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "8d90d61da62549208fe117b4384a469b" candidate_term: 6 candidate_status { last_received { term: 5 index: 710 } } ignore_live_leader: false dest_uuid: "42895a9bcba64aa484f6509605035d02" is_pre_election: true
I20260812 06:36:25.454610  6703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "2df4308c852240859ada53d216ab69ae" candidate_term: 6 candidate_status { last_received { term: 5 index: 757 } } ignore_live_leader: false dest_uuid: "42895a9bcba64aa484f6509605035d02" is_pre_election: true
I20260812 06:36:25.454831  6693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "2df4308c852240859ada53d216ab69ae" candidate_term: 6 candidate_status { last_received { term: 5 index: 757 } } ignore_live_leader: false dest_uuid: "42895a9bcba64aa484f6509605035d02" is_pre_election: true
I20260812 06:36:25.455116  6693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "2df4308c852240859ada53d216ab69ae" candidate_term: 8 candidate_status { last_received { term: 5 index: 757 } } ignore_live_leader: false dest_uuid: "42895a9bcba64aa484f6509605035d02"
W20260812 06:36:25.455268  6597 connection.cc:256] RPC call timeout handler was delayed by -3.147953512147069s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1s for a response
W20260812 06:36:25.455325  6597 connection.cc:256] RPC call timeout handler was delayed by -2.9234571191482246s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1s for a response
I20260812 06:36:25.455345  6693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "2df4308c852240859ada53d216ab69ae" candidate_term: 6 candidate_status { last_received { term: 5 index: 757 } } ignore_live_leader: false dest_uuid: "42895a9bcba64aa484f6509605035d02"
I20260812 06:36:25.455430  6700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "222b20de5a1145c6b8a11df8374e8790" candidate_term: 6 candidate_status { last_received { term: 5 index: 757 } } ignore_live_leader: false dest_uuid: "42895a9bcba64aa484f6509605035d02" is_pre_election: true
I20260812 06:36:25.455543  6693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" candidate_term: 6 candidate_status { last_received { term: 5 index: 630 } } ignore_live_leader: false dest_uuid: "42895a9bcba64aa484f6509605035d02" is_pre_election: true
I20260812 06:36:25.455665  6700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" candidate_term: 6 candidate_status { last_received { term: 5 index: 630 } } ignore_live_leader: false dest_uuid: "42895a9bcba64aa484f6509605035d02" is_pre_election: true
I20260812 06:36:25.455713  6693 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "2df4308c852240859ada53d216ab69ae" candidate_term: 8 candidate_status { last_received { term: 5 index: 757 } } ignore_live_leader: false dest_uuid: "42895a9bcba64aa484f6509605035d02" is_pre_election: true
I20260812 06:36:25.455956  6699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "222b20de5a1145c6b8a11df8374e8790" candidate_term: 6 candidate_status { last_received { term: 5 index: 757 } } ignore_live_leader: false dest_uuid: "42895a9bcba64aa484f6509605035d02"
I20260812 06:36:25.456193  6696 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "222b20de5a1145c6b8a11df8374e8790" candidate_term: 6 candidate_status { last_received { term: 5 index: 757 } } ignore_live_leader: false dest_uuid: "42895a9bcba64aa484f6509605035d02" is_pre_election: true
I20260812 06:36:25.456382  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" candidate_term: 6 candidate_status { last_received { term: 5 index: 665 } } ignore_live_leader: false dest_uuid: "42895a9bcba64aa484f6509605035d02" is_pre_election: true
I20260812 06:36:25.456420  6694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "222b20de5a1145c6b8a11df8374e8790" candidate_term: 6 candidate_status { last_received { term: 5 index: 757 } } ignore_live_leader: false dest_uuid: "42895a9bcba64aa484f6509605035d02"
I20260812 06:36:25.456588  6680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" candidate_term: 7 candidate_status { last_received { term: 5 index: 753 } } ignore_live_leader: false dest_uuid: "42895a9bcba64aa484f6509605035d02"
I20260812 06:36:25.456631  6691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" candidate_term: 6 candidate_status { last_received { term: 5 index: 753 } } ignore_live_leader: false dest_uuid: "42895a9bcba64aa484f6509605035d02" is_pre_election: true
I20260812 06:36:25.457091  6686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" candidate_term: 7 candidate_status { last_received { term: 5 index: 753 } } ignore_live_leader: false dest_uuid: "42895a9bcba64aa484f6509605035d02" is_pre_election: true
I20260812 06:36:25.457280  6821 leader_election.cc:304] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 7 responses out of 7 voters: 3 yes votes; 4 no votes. yes voters: 222b20de5a1145c6b8a11df8374e8790, b037d17b21f8422fa5e41b70eb4dcd08, ec6c8d818e744cdbb6b0d15b321010e0; no voters: 2df4308c852240859ada53d216ab69ae, 42895a9bcba64aa484f6509605035d02, 7962f587ccd842ad8a3f6ed9a5d7707e, 8d90d61da62549208fe117b4384a469b
I20260812 06:36:25.457401  6672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" candidate_term: 6 candidate_status { last_received { term: 5 index: 665 } } ignore_live_leader: false dest_uuid: "42895a9bcba64aa484f6509605035d02" is_pre_election: true
I20260812 06:36:25.454842  6703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "2df4308c852240859ada53d216ab69ae" candidate_term: 6 candidate_status { last_received { term: 5 index: 757 } } ignore_live_leader: false dest_uuid: "42895a9bcba64aa484f6509605035d02"
I20260812 06:36:25.455449  6701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "8d90d61da62549208fe117b4384a469b" candidate_term: 6 candidate_status { last_received { term: 5 index: 710 } } ignore_live_leader: false dest_uuid: "42895a9bcba64aa484f6509605035d02" is_pre_election: true
I20260812 06:36:25.457994  7963 raft_consensus.cc:2749] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [term 8 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
W20260812 06:36:25.455358  6597 connection.cc:256] RPC call timeout handler was delayed by -2.8149254831951112s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1s for a response
I20260812 06:36:25.459004  6919 raft_consensus.cc:1240] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [term 8 FOLLOWER]: Rejecting Update request from peer 42895a9bcba64aa484f6509605035d02 for earlier term 5. Current term is 8. Ops: []
I20260812 06:36:25.460296  7108 raft_consensus.cc:1240] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [term 8 LEADER]: Rejecting Update request from peer 42895a9bcba64aa484f6509605035d02 for earlier term 5. Current term is 8. Ops: []
I20260812 06:36:25.460405  7883 consensus_queue.cc:1059] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "222b20de5a1145c6b8a11df8374e8790" member_type: VOTER last_known_addr { host: "127.4.177.5" port: 43521 }, Status: INVALID_TERM, Last received: 5.757, Next index: 758, Last known committed idx: 749, Time since last communication: 0.000s
I20260812 06:36:25.460556  7883 raft_consensus.cc:3055] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [term 5 LEADER]: Stepping down as leader of term 5
I20260812 06:36:25.460608  7883 raft_consensus.cc:740] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 42895a9bcba64aa484f6509605035d02, State: Running, Role: LEADER
I20260812 06:36:25.460758  7883 consensus_queue.cc:260] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 753, Committed index: 753, Last appended: 5.757, Last appended by leader: 757, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df4308c852240859ada53d216ab69ae" member_type: VOTER last_known_addr { host: "127.4.177.6" port: 34269 } } peers { permanent_uuid: "42895a9bcba64aa484f6509605035d02" member_type: VOTER last_known_addr { host: "127.4.177.4" port: 38181 } } peers { permanent_uuid: "222b20de5a1145c6b8a11df8374e8790" member_type: VOTER last_known_addr { host: "127.4.177.5" port: 43521 } } peers { permanent_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 40897 } } peers { permanent_uuid: "8d90d61da62549208fe117b4384a469b" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 41539 } } peers { permanent_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" member_type: VOTER last_known_addr { host: "127.4.177.7" port: 46299 } } peers { permanent_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 41057 } }
I20260812 06:36:25.460927  6480 raft_consensus.cc:1240] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [term 7 LEADER]: Rejecting Update request from peer 42895a9bcba64aa484f6509605035d02 for earlier term 5. Current term is 7. Ops: [5.754-5.757]
I20260812 06:36:25.461607  7879 consensus_queue.cc:1059] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2df4308c852240859ada53d216ab69ae" member_type: VOTER last_known_addr { host: "127.4.177.6" port: 34269 }, Status: INVALID_TERM, Last received: 5.757, Next index: 758, Last known committed idx: 749, Time since last communication: 0.000s
I20260812 06:36:25.461714  7879 consensus_queue.cc:1243] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [NON_LEADER]: Peer 2df4308c852240859ada53d216ab69ae log is divergent from this leader: its last log entry 8.758 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 749
I20260812 06:36:25.461789  7964 consensus_queue.cc:1059] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 40897 }, Status: INVALID_TERM, Last received: 5.753, Next index: 754, Last known committed idx: 747, Time since last communication: 0.000s
I20260812 06:36:25.461848  7971 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:42408 (ReqId={client: 3067f3aef2a44cc4bb7f2ccd6acf83db, seq_no=746, attempt_no=0}) took 3729 ms. Trace:
I20260812 06:36:25.461915  7964 consensus_queue.cc:1243] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [NON_LEADER]: Peer 7962f587ccd842ad8a3f6ed9a5d7707e log is divergent from this leader: its last log entry 7.754 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 747
I20260812 06:36:25.462584  7970 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:42408 (ReqId={client: 3067f3aef2a44cc4bb7f2ccd6acf83db, seq_no=747, attempt_no=0}) took 3726 ms. Trace:
I20260812 06:36:25.461930  7971 rpcz_store.cc:276] 0812 06:36:21.732065 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 06:36:21.732111 (+    46us) service_pool.cc:224] Handling call
0812 06:36:25.461827 (+3729716us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0812 06:36:21.734405 (+     0us) write_op.cc:183] PREPARE: starting on tablet 7065bb403e3047dd9a0853c49318ef1f
0812 06:36:21.734471 (+    66us) write_op.cc:432] Acquiring schema lock in shared mode
0812 06:36:21.734475 (+     4us) write_op.cc:435] Acquired schema lock
0812 06:36:21.734476 (+     1us) tablet.cc:662] Decoding operations
0812 06:36:21.734565 (+    89us) write_op.cc:620] Acquiring the partition lock for write op
0812 06:36:21.734571 (+     6us) write_op.cc:641] Partition lock acquired for write op
0812 06:36:21.734573 (+     2us) tablet.cc:685] Acquiring locks for 10 operations
0812 06:36:21.734630 (+    57us) tablet.cc:701] Row locks acquired
0812 06:36:21.734631 (+     1us) write_op.cc:260] PREPARE: finished
0812 06:36:21.734677 (+    46us) write_op.cc:270] Start()
0812 06:36:21.734696 (+    19us) write_op.cc:276] Timestamp: P: 1786516581734551 usec, L: 0
0812 06:36:21.734697 (+     1us) op_driver.cc:348] REPLICATION: starting
0812 06:36:21.734793 (+    96us) log.cc:844] Serialized 696 byte log entry
0812 06:36:25.459821 (+3725028us) op_driver.cc:464] REPLICATION: finished
0812 06:36:25.461072 (+  1251us) write_op.cc:301] APPLY: starting
0812 06:36:25.461093 (+    21us) tablet.cc:1367] starting BulkCheckPresence
0812 06:36:25.461137 (+    44us) tablet.cc:1370] finished BulkCheckPresence
0812 06:36:25.461151 (+    14us) tablet.cc:1372] starting ApplyRowOperation cycle
0812 06:36:25.461407 (+   256us) tablet.cc:1383] finished ApplyRowOperation cycle
0812 06:36:25.461444 (+    37us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0812 06:36:25.461449 (+     5us) write_op.cc:312] APPLY: finished
0812 06:36:25.461530 (+    81us) log.cc:844] Serialized 77 byte log entry
0812 06:36:25.461713 (+   183us) write_op.cc:489] Releasing partition, row and schema locks
0812 06:36:25.461735 (+    22us) write_op.cc:454] Released schema lock
0812 06:36:25.461744 (+     9us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":1182,"num_ops":10,"prepare.queue_time_us":2005,"prepare.run_cpu_time_us":423,"prepare.run_wall_time_us":423,"replication_time_us":3725089,"thread_start_us":185,"threads_started":2}]]}
I20260812 06:36:25.462642  7970 rpcz_store.cc:276] 0812 06:36:21.735951 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 06:36:21.735998 (+    47us) service_pool.cc:224] Handling call
0812 06:36:25.462558 (+3726560us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0812 06:36:21.736346 (+     0us) write_op.cc:183] PREPARE: starting on tablet 7065bb403e3047dd9a0853c49318ef1f
0812 06:36:21.736434 (+    88us) write_op.cc:432] Acquiring schema lock in shared mode
0812 06:36:21.736437 (+     3us) write_op.cc:435] Acquired schema lock
0812 06:36:21.736439 (+     2us) tablet.cc:662] Decoding operations
0812 06:36:21.736557 (+   118us) write_op.cc:620] Acquiring the partition lock for write op
0812 06:36:21.736563 (+     6us) write_op.cc:641] Partition lock acquired for write op
0812 06:36:21.736567 (+     4us) tablet.cc:685] Acquiring locks for 10 operations
0812 06:36:21.736637 (+    70us) tablet.cc:701] Row locks acquired
0812 06:36:21.736639 (+     2us) write_op.cc:260] PREPARE: finished
0812 06:36:21.736670 (+    31us) write_op.cc:270] Start()
0812 06:36:21.736690 (+    20us) write_op.cc:276] Timestamp: P: 1786516581736543 usec, L: 0
0812 06:36:21.736692 (+     2us) op_driver.cc:348] REPLICATION: starting
0812 06:36:21.736806 (+   114us) log.cc:844] Serialized 696 byte log entry
0812 06:36:25.459983 (+3723177us) op_driver.cc:464] REPLICATION: finished
0812 06:36:25.462045 (+  2062us) write_op.cc:301] APPLY: starting
0812 06:36:25.462056 (+    11us) tablet.cc:1367] starting BulkCheckPresence
0812 06:36:25.462085 (+    29us) tablet.cc:1370] finished BulkCheckPresence
0812 06:36:25.462097 (+    12us) tablet.cc:1372] starting ApplyRowOperation cycle
0812 06:36:25.462353 (+   256us) tablet.cc:1383] finished ApplyRowOperation cycle
0812 06:36:25.462380 (+    27us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0812 06:36:25.462384 (+     4us) write_op.cc:312] APPLY: finished
0812 06:36:25.462448 (+    64us) log.cc:844] Serialized 77 byte log entry
0812 06:36:25.462485 (+    37us) write_op.cc:489] Releasing partition, row and schema locks
0812 06:36:25.462498 (+    13us) write_op.cc:454] Released schema lock
0812 06:36:25.462502 (+     4us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":2013,"num_ops":10,"prepare.queue_time_us":59,"prepare.run_cpu_time_us":515,"prepare.run_wall_time_us":721,"replication_time_us":3723268,"thread_start_us":95,"threads_started":1}]]}
I20260812 06:36:25.462931  7883 raft_consensus.cc:3060] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [term 5 FOLLOWER]: Advancing to term 8
I20260812 06:36:25.464280  7970 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:42408 (ReqId={client: 3067f3aef2a44cc4bb7f2ccd6acf83db, seq_no=748, attempt_no=0}) took 3727 ms. Trace:
I20260812 06:36:25.464339  7970 rpcz_store.cc:276] 0812 06:36:21.736622 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 06:36:21.736732 (+   110us) service_pool.cc:224] Handling call
0812 06:36:25.464271 (+3727539us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0812 06:36:21.737089 (+     0us) write_op.cc:183] PREPARE: starting on tablet 7065bb403e3047dd9a0853c49318ef1f
0812 06:36:21.737163 (+    74us) write_op.cc:432] Acquiring schema lock in shared mode
0812 06:36:21.737167 (+     4us) write_op.cc:435] Acquired schema lock
0812 06:36:21.737168 (+     1us) tablet.cc:662] Decoding operations
0812 06:36:21.737257 (+    89us) write_op.cc:620] Acquiring the partition lock for write op
0812 06:36:21.737263 (+     6us) write_op.cc:641] Partition lock acquired for write op
0812 06:36:21.737266 (+     3us) tablet.cc:685] Acquiring locks for 10 operations
0812 06:36:21.737329 (+    63us) tablet.cc:701] Row locks acquired
0812 06:36:21.737331 (+     2us) write_op.cc:260] PREPARE: finished
0812 06:36:21.737359 (+    28us) write_op.cc:270] Start()
0812 06:36:21.737375 (+    16us) write_op.cc:276] Timestamp: P: 1786516581737232 usec, L: 0
0812 06:36:21.737377 (+     2us) op_driver.cc:348] REPLICATION: starting
0812 06:36:21.737489 (+   112us) log.cc:844] Serialized 696 byte log entry
0812 06:36:25.460139 (+3722650us) op_driver.cc:464] REPLICATION: finished
0812 06:36:25.462990 (+  2851us) write_op.cc:301] APPLY: starting
0812 06:36:25.462999 (+     9us) tablet.cc:1367] starting BulkCheckPresence
0812 06:36:25.463026 (+    27us) tablet.cc:1370] finished BulkCheckPresence
0812 06:36:25.463036 (+    10us) tablet.cc:1372] starting ApplyRowOperation cycle
0812 06:36:25.464049 (+  1013us) tablet.cc:1383] finished ApplyRowOperation cycle
0812 06:36:25.464080 (+    31us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0812 06:36:25.464085 (+     5us) write_op.cc:312] APPLY: finished
0812 06:36:25.464160 (+    75us) log.cc:844] Serialized 77 byte log entry
0812 06:36:25.464200 (+    40us) write_op.cc:489] Releasing partition, row and schema locks
0812 06:36:25.464212 (+    12us) write_op.cc:454] Released schema lock
0812 06:36:25.464217 (+     5us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":2819,"num_ops":10,"prepare.queue_time_us":90,"prepare.run_cpu_time_us":449,"prepare.run_wall_time_us":639,"replication_time_us":3722745,"thread_start_us":65,"threads_started":1}]]}
W20260812 06:36:25.464676  6621 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:42408: Illegal state: replica 42895a9bcba64aa484f6509605035d02 is not leader of this config: current role FOLLOWER
I20260812 06:36:25.465085  7970 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:42408 (ReqId={client: 3067f3aef2a44cc4bb7f2ccd6acf83db, seq_no=749, attempt_no=0}) took 3728 ms. Trace:
I20260812 06:36:25.465137  7970 rpcz_store.cc:276] 0812 06:36:21.736690 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 06:36:21.736859 (+   169us) service_pool.cc:224] Handling call
0812 06:36:25.465075 (+3728216us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0812 06:36:21.737756 (+     0us) write_op.cc:183] PREPARE: starting on tablet 7065bb403e3047dd9a0853c49318ef1f
0812 06:36:21.737830 (+    74us) write_op.cc:432] Acquiring schema lock in shared mode
0812 06:36:21.737834 (+     4us) write_op.cc:435] Acquired schema lock
0812 06:36:21.737835 (+     1us) tablet.cc:662] Decoding operations
0812 06:36:21.737918 (+    83us) write_op.cc:620] Acquiring the partition lock for write op
0812 06:36:21.737923 (+     5us) write_op.cc:641] Partition lock acquired for write op
0812 06:36:21.737926 (+     3us) tablet.cc:685] Acquiring locks for 10 operations
0812 06:36:21.738012 (+    86us) tablet.cc:701] Row locks acquired
0812 06:36:21.738014 (+     2us) write_op.cc:260] PREPARE: finished
0812 06:36:21.738042 (+    28us) write_op.cc:270] Start()
0812 06:36:21.738060 (+    18us) write_op.cc:276] Timestamp: P: 1786516581737916 usec, L: 0
0812 06:36:21.738062 (+     2us) op_driver.cc:348] REPLICATION: starting
0812 06:36:21.738166 (+   104us) log.cc:844] Serialized 696 byte log entry
0812 06:36:25.460253 (+3722087us) op_driver.cc:464] REPLICATION: finished
0812 06:36:25.464672 (+  4419us) write_op.cc:301] APPLY: starting
0812 06:36:25.464681 (+     9us) tablet.cc:1367] starting BulkCheckPresence
0812 06:36:25.464714 (+    33us) tablet.cc:1370] finished BulkCheckPresence
0812 06:36:25.464726 (+    12us) tablet.cc:1372] starting ApplyRowOperation cycle
0812 06:36:25.464875 (+   149us) tablet.cc:1383] finished ApplyRowOperation cycle
0812 06:36:25.464905 (+    30us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0812 06:36:25.464911 (+     6us) write_op.cc:312] APPLY: finished
0812 06:36:25.464970 (+    59us) log.cc:844] Serialized 77 byte log entry
0812 06:36:25.465006 (+    36us) write_op.cc:489] Releasing partition, row and schema locks
0812 06:36:25.465017 (+    11us) write_op.cc:454] Released schema lock
0812 06:36:25.465022 (+     5us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":4388,"num_ops":10,"prepare.queue_time_us":624,"prepare.run_cpu_time_us":460,"prepare.run_wall_time_us":683,"replication_time_us":3722177,"thread_start_us":68,"threads_started":1}]]}
W20260812 06:36:25.466796  6626 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:42408: Illegal state: replica 42895a9bcba64aa484f6509605035d02 is not leader of this config: current role FOLLOWER
I20260812 06:36:25.466928  6047 raft_consensus.cc:1240] T 7065bb403e3047dd9a0853c49318ef1f P ec6c8d818e744cdbb6b0d15b321010e0 [term 8 FOLLOWER]: Rejecting Update request from peer 42895a9bcba64aa484f6509605035d02 for earlier term 5. Current term is 8. Ops: [5.631-5.757]
W20260812 06:36:25.468334  6621 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:42408: Illegal state: replica 42895a9bcba64aa484f6509605035d02 is not leader of this config: current role FOLLOWER
I20260812 06:36:25.468596  6247 raft_consensus.cc:1240] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b [term 8 FOLLOWER]: Rejecting Update request from peer 42895a9bcba64aa484f6509605035d02 for earlier term 5. Current term is 8. Ops: [5.711-5.757]
W20260812 06:36:25.469099  6626 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:42408: Illegal state: replica 42895a9bcba64aa484f6509605035d02 is not leader of this config: current role FOLLOWER
I20260812 06:36:25.470871  7321 raft_consensus.cc:1275] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08 [term 8 FOLLOWER]: Refusing update from remote peer 2df4308c852240859ada53d216ab69ae: Log matching property violated. Preceding OpId in replica: term: 5 index: 665. Preceding OpId from leader: term: 8 index: 759. (index mismatch)
I20260812 06:36:25.470963  6480 raft_consensus.cc:3055] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [term 7 LEADER]: Stepping down as leader of term 7
I20260812 06:36:25.471016  6247 raft_consensus.cc:1275] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b [term 8 FOLLOWER]: Refusing update from remote peer 2df4308c852240859ada53d216ab69ae: Log matching property violated. Preceding OpId in replica: term: 5 index: 710. Preceding OpId from leader: term: 8 index: 759. (index mismatch)
I20260812 06:36:25.471031  6480 raft_consensus.cc:740] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 7962f587ccd842ad8a3f6ed9a5d7707e, State: Running, Role: LEADER
I20260812 06:36:25.471202  6480 consensus_queue.cc:260] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 747, Committed index: 747, Last appended: 7.754, Last appended by leader: 754, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df4308c852240859ada53d216ab69ae" member_type: VOTER last_known_addr { host: "127.4.177.6" port: 34269 } } peers { permanent_uuid: "42895a9bcba64aa484f6509605035d02" member_type: VOTER last_known_addr { host: "127.4.177.4" port: 38181 } } peers { permanent_uuid: "222b20de5a1145c6b8a11df8374e8790" member_type: VOTER last_known_addr { host: "127.4.177.5" port: 43521 } } peers { permanent_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 40897 } } peers { permanent_uuid: "8d90d61da62549208fe117b4384a469b" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 41539 } } peers { permanent_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" member_type: VOTER last_known_addr { host: "127.4.177.7" port: 46299 } } peers { permanent_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 41057 } }
I20260812 06:36:25.471359  7959 consensus_queue.cc:1048] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" member_type: VOTER last_known_addr { host: "127.4.177.7" port: 46299 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 758, Last known committed idx: 662, Time since last communication: 0.000s
I20260812 06:36:25.471487  6480 raft_consensus.cc:3060] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [term 7 FOLLOWER]: Advancing to term 8
I20260812 06:36:25.471887  6047 raft_consensus.cc:1275] T 7065bb403e3047dd9a0853c49318ef1f P ec6c8d818e744cdbb6b0d15b321010e0 [term 8 FOLLOWER]: Refusing update from remote peer 2df4308c852240859ada53d216ab69ae: Log matching property violated. Preceding OpId in replica: term: 5 index: 630. Preceding OpId from leader: term: 8 index: 759. (index mismatch)
I20260812 06:36:25.472525  7958 consensus_queue.cc:1048] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "8d90d61da62549208fe117b4384a469b" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 41539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 758, Last known committed idx: 702, Time since last communication: 0.000s
I20260812 06:36:25.472741  6480 raft_consensus.cc:1275] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [term 8 FOLLOWER]: Refusing update from remote peer 2df4308c852240859ada53d216ab69ae: Log matching property violated. Preceding OpId in replica: term: 7 index: 754. Preceding OpId from leader: term: 8 index: 759. (index mismatch)
I20260812 06:36:25.472990  7959 consensus_queue.cc:1048] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 41057 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 758, Last known committed idx: 622, Time since last communication: 0.000s
I20260812 06:36:25.473186  6701 raft_consensus.cc:1275] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [term 8 FOLLOWER]: Refusing update from remote peer 2df4308c852240859ada53d216ab69ae: Log matching property violated. Preceding OpId in replica: term: 5 index: 757. Preceding OpId from leader: term: 8 index: 759. (index mismatch)
I20260812 06:36:25.474052  6919 raft_consensus.cc:1275] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [term 8 FOLLOWER]: Refusing update from remote peer 2df4308c852240859ada53d216ab69ae: Log matching property violated. Preceding OpId in replica: term: 5 index: 757. Preceding OpId from leader: term: 8 index: 760. (index mismatch)
I20260812 06:36:25.474529  7959 consensus_queue.cc:1048] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "42895a9bcba64aa484f6509605035d02" member_type: VOTER last_known_addr { host: "127.4.177.4" port: 38181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 758, Last known committed idx: 753, Time since last communication: 0.000s
I20260812 06:36:25.474815  7959 consensus_queue.cc:1048] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 40897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 758, Last known committed idx: 747, Time since last communication: 0.000s
I20260812 06:36:25.474902  7959 consensus_queue.cc:1243] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [LEADER]: Peer 7962f587ccd842ad8a3f6ed9a5d7707e log is divergent from this leader: its last log entry 7.754 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 747
I20260812 06:36:25.475135  7959 consensus_queue.cc:1048] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "222b20de5a1145c6b8a11df8374e8790" member_type: VOTER last_known_addr { host: "127.4.177.5" port: 43521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 758, Last known committed idx: 749, Time since last communication: 0.000s
I20260812 06:36:25.476213  6480 raft_consensus.cc:1217] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [term 8 FOLLOWER]: Deduplicated request from leader. Original: 5.747->[5.748-8.760]   Dedup: 5.753->[5.754-8.760]
I20260812 06:36:25.476318  6480 pending_rounds.cc:85] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e: Aborting all ops after (but not including) 753
I20260812 06:36:25.476352  6480 pending_rounds.cc:107] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e: Aborting uncommitted NO_OP operation due to leader change: 7.754
I20260812 06:36:25.476380  6480 raft_consensus.cc:2889] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260812 06:36:25.488070  7967 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:42408 (ReqId={client: 3067f3aef2a44cc4bb7f2ccd6acf83db, seq_no=750, attempt_no=0}) took 3743 ms. Trace:
I20260812 06:36:25.488183  7967 rpcz_store.cc:276] 0812 06:36:21.744754 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 06:36:21.744822 (+    68us) service_pool.cc:224] Handling call
0812 06:36:25.488050 (+3743228us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0812 06:36:21.745228 (+     0us) write_op.cc:183] PREPARE: starting on tablet 7065bb403e3047dd9a0853c49318ef1f
0812 06:36:21.745301 (+    73us) write_op.cc:432] Acquiring schema lock in shared mode
0812 06:36:21.745304 (+     3us) write_op.cc:435] Acquired schema lock
0812 06:36:21.745306 (+     2us) tablet.cc:662] Decoding operations
0812 06:36:21.745395 (+    89us) write_op.cc:620] Acquiring the partition lock for write op
0812 06:36:21.745400 (+     5us) write_op.cc:641] Partition lock acquired for write op
0812 06:36:21.745403 (+     3us) tablet.cc:685] Acquiring locks for 10 operations
0812 06:36:21.745610 (+   207us) tablet.cc:701] Row locks acquired
0812 06:36:21.745612 (+     2us) write_op.cc:260] PREPARE: finished
0812 06:36:21.745635 (+    23us) write_op.cc:270] Start()
0812 06:36:21.745653 (+    18us) write_op.cc:276] Timestamp: P: 1786516581745510 usec, L: 0
0812 06:36:21.745655 (+     2us) op_driver.cc:348] REPLICATION: starting
0812 06:36:21.745751 (+    96us) log.cc:844] Serialized 696 byte log entry
0812 06:36:25.487042 (+3741291us) op_driver.cc:464] REPLICATION: finished
0812 06:36:25.487579 (+   537us) write_op.cc:301] APPLY: starting
0812 06:36:25.487592 (+    13us) tablet.cc:1367] starting BulkCheckPresence
0812 06:36:25.487627 (+    35us) tablet.cc:1370] finished BulkCheckPresence
0812 06:36:25.487643 (+    16us) tablet.cc:1372] starting ApplyRowOperation cycle
0812 06:36:25.487824 (+   181us) tablet.cc:1383] finished ApplyRowOperation cycle
0812 06:36:25.487856 (+    32us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0812 06:36:25.487860 (+     4us) write_op.cc:312] APPLY: finished
0812 06:36:25.487921 (+    61us) log.cc:844] Serialized 77 byte log entry
0812 06:36:25.487963 (+    42us) write_op.cc:489] Releasing partition, row and schema locks
0812 06:36:25.487978 (+    15us) write_op.cc:454] Released schema lock
0812 06:36:25.487983 (+     5us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":487,"num_ops":10,"prepare.queue_time_us":128,"prepare.run_cpu_time_us":444,"prepare.run_wall_time_us":574,"replication_time_us":3741361}]]}
I20260812 06:36:25.488981  7967 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:42408 (ReqId={client: 3067f3aef2a44cc4bb7f2ccd6acf83db, seq_no=751, attempt_no=0}) took 3744 ms. Trace:
I20260812 06:36:25.489046  7967 rpcz_store.cc:276] 0812 06:36:21.744888 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 06:36:21.745048 (+   160us) service_pool.cc:224] Handling call
0812 06:36:25.488968 (+3743920us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0812 06:36:21.745826 (+     0us) write_op.cc:183] PREPARE: starting on tablet 7065bb403e3047dd9a0853c49318ef1f
0812 06:36:21.745906 (+    80us) write_op.cc:432] Acquiring schema lock in shared mode
0812 06:36:21.745909 (+     3us) write_op.cc:435] Acquired schema lock
0812 06:36:21.745910 (+     1us) tablet.cc:662] Decoding operations
0812 06:36:21.747087 (+  1177us) write_op.cc:620] Acquiring the partition lock for write op
0812 06:36:21.747093 (+     6us) write_op.cc:641] Partition lock acquired for write op
0812 06:36:21.747096 (+     3us) tablet.cc:685] Acquiring locks for 10 operations
0812 06:36:21.747156 (+    60us) tablet.cc:701] Row locks acquired
0812 06:36:21.747158 (+     2us) write_op.cc:260] PREPARE: finished
0812 06:36:21.747303 (+   145us) write_op.cc:270] Start()
0812 06:36:21.747323 (+    20us) write_op.cc:276] Timestamp: P: 1786516581747058 usec, L: 0
0812 06:36:21.747324 (+     1us) op_driver.cc:348] REPLICATION: starting
0812 06:36:21.747406 (+    82us) log.cc:844] Serialized 686 byte log entry
0812 06:36:25.487110 (+3739704us) op_driver.cc:464] REPLICATION: finished
0812 06:36:25.488551 (+  1441us) write_op.cc:301] APPLY: starting
0812 06:36:25.488561 (+    10us) tablet.cc:1367] starting BulkCheckPresence
0812 06:36:25.488589 (+    28us) tablet.cc:1370] finished BulkCheckPresence
0812 06:36:25.488599 (+    10us) tablet.cc:1372] starting ApplyRowOperation cycle
0812 06:36:25.488777 (+   178us) tablet.cc:1383] finished ApplyRowOperation cycle
0812 06:36:25.488802 (+    25us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0812 06:36:25.488807 (+     5us) write_op.cc:312] APPLY: finished
0812 06:36:25.488859 (+    52us) log.cc:844] Serialized 77 byte log entry
0812 06:36:25.488893 (+    34us) write_op.cc:489] Releasing partition, row and schema locks
0812 06:36:25.488906 (+    13us) write_op.cc:454] Released schema lock
0812 06:36:25.488911 (+     5us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":1412,"num_ops":10,"prepare.queue_time_us":555,"prepare.run_cpu_time_us":449,"prepare.run_wall_time_us":1642,"replication_time_us":3739772}]]}
I20260812 06:36:25.489989  7967 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:42408 (ReqId={client: 3067f3aef2a44cc4bb7f2ccd6acf83db, seq_no=753, attempt_no=0}) took 3745 ms. Trace:
I20260812 06:36:25.490742  7971 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:42408 (ReqId={client: 3067f3aef2a44cc4bb7f2ccd6acf83db, seq_no=752, attempt_no=0}) took 3745 ms. Trace:
I20260812 06:36:25.490803  7971 rpcz_store.cc:276] 0812 06:36:21.745011 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 06:36:21.745545 (+   534us) service_pool.cc:224] Handling call
0812 06:36:25.490709 (+3745164us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0812 06:36:21.747959 (+     0us) write_op.cc:183] PREPARE: starting on tablet 7065bb403e3047dd9a0853c49318ef1f
0812 06:36:21.748032 (+    73us) write_op.cc:432] Acquiring schema lock in shared mode
0812 06:36:21.748038 (+     6us) write_op.cc:435] Acquired schema lock
0812 06:36:21.748041 (+     3us) tablet.cc:662] Decoding operations
0812 06:36:21.748128 (+    87us) write_op.cc:620] Acquiring the partition lock for write op
0812 06:36:21.748133 (+     5us) write_op.cc:641] Partition lock acquired for write op
0812 06:36:21.748137 (+     4us) tablet.cc:685] Acquiring locks for 10 operations
0812 06:36:21.748190 (+    53us) tablet.cc:701] Row locks acquired
0812 06:36:21.748191 (+     1us) write_op.cc:260] PREPARE: finished
0812 06:36:21.748214 (+    23us) write_op.cc:270] Start()
0812 06:36:21.748234 (+    20us) write_op.cc:276] Timestamp: P: 1786516581748088 usec, L: 0
0812 06:36:21.748237 (+     3us) op_driver.cc:348] REPLICATION: starting
0812 06:36:21.748333 (+    96us) log.cc:844] Serialized 676 byte log entry
0812 06:36:25.487187 (+3738854us) op_driver.cc:464] REPLICATION: finished
0812 06:36:25.490279 (+  3092us) write_op.cc:301] APPLY: starting
0812 06:36:25.490295 (+    16us) tablet.cc:1367] starting BulkCheckPresence
0812 06:36:25.490322 (+    27us) tablet.cc:1370] finished BulkCheckPresence
0812 06:36:25.490335 (+    13us) tablet.cc:1372] starting ApplyRowOperation cycle
0812 06:36:25.490530 (+   195us) tablet.cc:1383] finished ApplyRowOperation cycle
0812 06:36:25.490554 (+    24us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0812 06:36:25.490559 (+     5us) write_op.cc:312] APPLY: finished
0812 06:36:25.490611 (+    52us) log.cc:844] Serialized 77 byte log entry
0812 06:36:25.490646 (+    35us) write_op.cc:489] Releasing partition, row and schema locks
0812 06:36:25.490658 (+    12us) write_op.cc:454] Released schema lock
0812 06:36:25.490662 (+     4us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":3050,"num_ops":10,"prepare.queue_time_us":2213,"prepare.run_cpu_time_us":444,"prepare.run_wall_time_us":444,"replication_time_us":3738937}]]}
I20260812 06:36:25.490054  7967 rpcz_store.cc:276] 0812 06:36:21.744955 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 06:36:21.745291 (+   336us) service_pool.cc:224] Handling call
0812 06:36:25.489978 (+3744687us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0812 06:36:21.747496 (+     0us) write_op.cc:183] PREPARE: starting on tablet 7065bb403e3047dd9a0853c49318ef1f
0812 06:36:21.747574 (+    78us) write_op.cc:432] Acquiring schema lock in shared mode
0812 06:36:21.747577 (+     3us) write_op.cc:435] Acquired schema lock
0812 06:36:21.747578 (+     1us) tablet.cc:662] Decoding operations
0812 06:36:21.747663 (+    85us) write_op.cc:620] Acquiring the partition lock for write op
0812 06:36:21.747669 (+     6us) write_op.cc:641] Partition lock acquired for write op
0812 06:36:21.747672 (+     3us) tablet.cc:685] Acquiring locks for 10 operations
0812 06:36:21.747722 (+    50us) tablet.cc:701] Row locks acquired
0812 06:36:21.747723 (+     1us) write_op.cc:260] PREPARE: finished
0812 06:36:21.747745 (+    22us) write_op.cc:270] Start()
0812 06:36:21.747759 (+    14us) write_op.cc:276] Timestamp: P: 1786516581747620 usec, L: 0
0812 06:36:21.747760 (+     1us) op_driver.cc:348] REPLICATION: starting
0812 06:36:21.747866 (+   106us) log.cc:844] Serialized 696 byte log entry
0812 06:36:25.487150 (+3739284us) op_driver.cc:464] REPLICATION: finished
0812 06:36:25.489576 (+  2426us) write_op.cc:301] APPLY: starting
0812 06:36:25.489588 (+    12us) tablet.cc:1367] starting BulkCheckPresence
0812 06:36:25.489615 (+    27us) tablet.cc:1370] finished BulkCheckPresence
0812 06:36:25.489628 (+    13us) tablet.cc:1372] starting ApplyRowOperation cycle
0812 06:36:25.489801 (+   173us) tablet.cc:1383] finished ApplyRowOperation cycle
0812 06:36:25.489826 (+    25us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0812 06:36:25.489830 (+     4us) write_op.cc:312] APPLY: finished
0812 06:36:25.489882 (+    52us) log.cc:844] Serialized 77 byte log entry
0812 06:36:25.489913 (+    31us) write_op.cc:489] Releasing partition, row and schema locks
0812 06:36:25.489925 (+    12us) write_op.cc:454] Released schema lock
0812 06:36:25.489929 (+     4us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":2392,"num_ops":10,"prepare.queue_time_us":1979,"prepare.run_cpu_time_us":429,"prepare.run_wall_time_us":428,"replication_time_us":3739378}]]}
W20260812 06:36:26.416821  8037 log.cc:927] Time spent T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08: Append to log took a long time: real 0.569s	user 0.000s	sys 0.000s
I20260812 06:36:27.579860  8046 raft_consensus.cc:493] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 2df4308c852240859ada53d216ab69ae)
I20260812 06:36:27.580029  8046 raft_consensus.cc:515] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df4308c852240859ada53d216ab69ae" member_type: VOTER last_known_addr { host: "127.4.177.6" port: 34269 } } peers { permanent_uuid: "42895a9bcba64aa484f6509605035d02" member_type: VOTER last_known_addr { host: "127.4.177.4" port: 38181 } } peers { permanent_uuid: "222b20de5a1145c6b8a11df8374e8790" member_type: VOTER last_known_addr { host: "127.4.177.5" port: 43521 } } peers { permanent_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 40897 } } peers { permanent_uuid: "8d90d61da62549208fe117b4384a469b" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 41539 } } peers { permanent_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" member_type: VOTER last_known_addr { host: "127.4.177.7" port: 46299 } } peers { permanent_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 41057 } }
I20260812 06:36:27.580672  8046 leader_election.cc:290] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 2df4308c852240859ada53d216ab69ae (127.4.177.6:34269), 42895a9bcba64aa484f6509605035d02 (127.4.177.4:38181), 7962f587ccd842ad8a3f6ed9a5d7707e (127.4.177.3:40897), 8d90d61da62549208fe117b4384a469b (127.4.177.2:41539), b037d17b21f8422fa5e41b70eb4dcd08 (127.4.177.7:46299), ec6c8d818e744cdbb6b0d15b321010e0 (127.4.177.1:41057)
I20260812 06:36:27.581110  7321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "222b20de5a1145c6b8a11df8374e8790" candidate_term: 9 candidate_status { last_received { term: 8 index: 935 } } ignore_live_leader: false dest_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" is_pre_election: true
I20260812 06:36:27.581157  6701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "222b20de5a1145c6b8a11df8374e8790" candidate_term: 9 candidate_status { last_received { term: 8 index: 935 } } ignore_live_leader: false dest_uuid: "42895a9bcba64aa484f6509605035d02" is_pre_election: true
I20260812 06:36:27.581166  6479 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "222b20de5a1145c6b8a11df8374e8790" candidate_term: 9 candidate_status { last_received { term: 8 index: 935 } } ignore_live_leader: false dest_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" is_pre_election: true
I20260812 06:36:27.581283  6701 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 222b20de5a1145c6b8a11df8374e8790 in term 8.
I20260812 06:36:27.581430  6247 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "222b20de5a1145c6b8a11df8374e8790" candidate_term: 9 candidate_status { last_received { term: 8 index: 935 } } ignore_live_leader: false dest_uuid: "8d90d61da62549208fe117b4384a469b" is_pre_election: true
I20260812 06:36:27.581573  6047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "222b20de5a1145c6b8a11df8374e8790" candidate_term: 9 candidate_status { last_received { term: 8 index: 935 } } ignore_live_leader: false dest_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" is_pre_election: true
I20260812 06:36:27.582033  6821 leader_election.cc:304] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 6 responses out of 7 voters: 2 yes votes; 4 no votes. yes voters: 222b20de5a1145c6b8a11df8374e8790, 42895a9bcba64aa484f6509605035d02; no voters: 7962f587ccd842ad8a3f6ed9a5d7707e, 8d90d61da62549208fe117b4384a469b, b037d17b21f8422fa5e41b70eb4dcd08, ec6c8d818e744cdbb6b0d15b321010e0
I20260812 06:36:27.582245  8046 raft_consensus.cc:2749] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260812 06:36:27.917163  8047 raft_consensus.cc:493] T 7065bb403e3047dd9a0853c49318ef1f P ec6c8d818e744cdbb6b0d15b321010e0 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 2df4308c852240859ada53d216ab69ae)
I20260812 06:36:27.917311  8047 raft_consensus.cc:515] T 7065bb403e3047dd9a0853c49318ef1f P ec6c8d818e744cdbb6b0d15b321010e0 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df4308c852240859ada53d216ab69ae" member_type: VOTER last_known_addr { host: "127.4.177.6" port: 34269 } } peers { permanent_uuid: "42895a9bcba64aa484f6509605035d02" member_type: VOTER last_known_addr { host: "127.4.177.4" port: 38181 } } peers { permanent_uuid: "222b20de5a1145c6b8a11df8374e8790" member_type: VOTER last_known_addr { host: "127.4.177.5" port: 43521 } } peers { permanent_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 40897 } } peers { permanent_uuid: "8d90d61da62549208fe117b4384a469b" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 41539 } } peers { permanent_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" member_type: VOTER last_known_addr { host: "127.4.177.7" port: 46299 } } peers { permanent_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 41057 } }
I20260812 06:36:27.917977  8047 leader_election.cc:290] T 7065bb403e3047dd9a0853c49318ef1f P ec6c8d818e744cdbb6b0d15b321010e0 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 2df4308c852240859ada53d216ab69ae (127.4.177.6:34269), 42895a9bcba64aa484f6509605035d02 (127.4.177.4:38181), 222b20de5a1145c6b8a11df8374e8790 (127.4.177.5:43521), 7962f587ccd842ad8a3f6ed9a5d7707e (127.4.177.3:40897), 8d90d61da62549208fe117b4384a469b (127.4.177.2:41539), b037d17b21f8422fa5e41b70eb4dcd08 (127.4.177.7:46299)
I20260812 06:36:27.918484  7321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" candidate_term: 9 candidate_status { last_received { term: 8 index: 805 } } ignore_live_leader: false dest_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" is_pre_election: true
I20260812 06:36:27.918541  6919 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" candidate_term: 9 candidate_status { last_received { term: 8 index: 805 } } ignore_live_leader: false dest_uuid: "222b20de5a1145c6b8a11df8374e8790" is_pre_election: true
I20260812 06:36:27.918678  6247 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" candidate_term: 9 candidate_status { last_received { term: 8 index: 805 } } ignore_live_leader: false dest_uuid: "8d90d61da62549208fe117b4384a469b" is_pre_election: true
I20260812 06:36:27.918741  6919 raft_consensus.cc:2410] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ec6c8d818e744cdbb6b0d15b321010e0 for term 9 because replica has last-logged OpId of term: 8 index: 935, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 805.
I20260812 06:36:27.918733  6479 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" candidate_term: 9 candidate_status { last_received { term: 8 index: 805 } } ignore_live_leader: false dest_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" is_pre_election: true
I20260812 06:36:27.919137  6701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" candidate_term: 9 candidate_status { last_received { term: 8 index: 805 } } ignore_live_leader: false dest_uuid: "42895a9bcba64aa484f6509605035d02" is_pre_election: true
I20260812 06:36:27.919250  5945 leader_election.cc:304] T 7065bb403e3047dd9a0853c49318ef1f P ec6c8d818e744cdbb6b0d15b321010e0 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 5 responses out of 7 voters: 1 yes votes; 4 no votes. yes voters: ec6c8d818e744cdbb6b0d15b321010e0; no voters: 222b20de5a1145c6b8a11df8374e8790, 7962f587ccd842ad8a3f6ed9a5d7707e, 8d90d61da62549208fe117b4384a469b, b037d17b21f8422fa5e41b70eb4dcd08
I20260812 06:36:27.919312  6701 raft_consensus.cc:2410] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ec6c8d818e744cdbb6b0d15b321010e0 for term 9 because replica has last-logged OpId of term: 8 index: 935, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 805.
I20260812 06:36:27.919525  8047 raft_consensus.cc:2749] T 7065bb403e3047dd9a0853c49318ef1f P ec6c8d818e744cdbb6b0d15b321010e0 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260812 06:36:27.921492  8048 raft_consensus.cc:493] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 2df4308c852240859ada53d216ab69ae)
I20260812 06:36:27.921607  8048 raft_consensus.cc:515] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df4308c852240859ada53d216ab69ae" member_type: VOTER last_known_addr { host: "127.4.177.6" port: 34269 } } peers { permanent_uuid: "42895a9bcba64aa484f6509605035d02" member_type: VOTER last_known_addr { host: "127.4.177.4" port: 38181 } } peers { permanent_uuid: "222b20de5a1145c6b8a11df8374e8790" member_type: VOTER last_known_addr { host: "127.4.177.5" port: 43521 } } peers { permanent_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 40897 } } peers { permanent_uuid: "8d90d61da62549208fe117b4384a469b" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 41539 } } peers { permanent_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" member_type: VOTER last_known_addr { host: "127.4.177.7" port: 46299 } } peers { permanent_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 41057 } }
I20260812 06:36:27.922492  8048 leader_election.cc:290] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 2df4308c852240859ada53d216ab69ae (127.4.177.6:34269), 42895a9bcba64aa484f6509605035d02 (127.4.177.4:38181), 222b20de5a1145c6b8a11df8374e8790 (127.4.177.5:43521), 8d90d61da62549208fe117b4384a469b (127.4.177.2:41539), b037d17b21f8422fa5e41b70eb4dcd08 (127.4.177.7:46299), ec6c8d818e744cdbb6b0d15b321010e0 (127.4.177.1:41057)
I20260812 06:36:27.922919  6919 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" candidate_term: 9 candidate_status { last_received { term: 8 index: 930 } } ignore_live_leader: false dest_uuid: "222b20de5a1145c6b8a11df8374e8790" is_pre_election: true
I20260812 06:36:27.922999  8049 raft_consensus.cc:493] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 2df4308c852240859ada53d216ab69ae)
I20260812 06:36:27.923071  6919 raft_consensus.cc:2410] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7962f587ccd842ad8a3f6ed9a5d7707e for term 9 because replica has last-logged OpId of term: 8 index: 935, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 930.
I20260812 06:36:27.923069  7321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" candidate_term: 9 candidate_status { last_received { term: 8 index: 930 } } ignore_live_leader: false dest_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" is_pre_election: true
I20260812 06:36:27.923097  8049 raft_consensus.cc:515] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df4308c852240859ada53d216ab69ae" member_type: VOTER last_known_addr { host: "127.4.177.6" port: 34269 } } peers { permanent_uuid: "42895a9bcba64aa484f6509605035d02" member_type: VOTER last_known_addr { host: "127.4.177.4" port: 38181 } } peers { permanent_uuid: "222b20de5a1145c6b8a11df8374e8790" member_type: VOTER last_known_addr { host: "127.4.177.5" port: 43521 } } peers { permanent_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 40897 } } peers { permanent_uuid: "8d90d61da62549208fe117b4384a469b" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 41539 } } peers { permanent_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" member_type: VOTER last_known_addr { host: "127.4.177.7" port: 46299 } } peers { permanent_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 41057 } }
I20260812 06:36:27.923107  6047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" candidate_term: 9 candidate_status { last_received { term: 8 index: 930 } } ignore_live_leader: false dest_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" is_pre_election: true
I20260812 06:36:27.923233  7321 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7962f587ccd842ad8a3f6ed9a5d7707e in term 8.
I20260812 06:36:27.923223  6047 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P ec6c8d818e744cdbb6b0d15b321010e0 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7962f587ccd842ad8a3f6ed9a5d7707e in term 8.
I20260812 06:36:27.923777  8049 leader_election.cc:290] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 2df4308c852240859ada53d216ab69ae (127.4.177.6:34269), 42895a9bcba64aa484f6509605035d02 (127.4.177.4:38181), 222b20de5a1145c6b8a11df8374e8790 (127.4.177.5:43521), 7962f587ccd842ad8a3f6ed9a5d7707e (127.4.177.3:40897), 8d90d61da62549208fe117b4384a469b (127.4.177.2:41539), ec6c8d818e744cdbb6b0d15b321010e0 (127.4.177.1:41057)
I20260812 06:36:27.923883  6247 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" candidate_term: 9 candidate_status { last_received { term: 8 index: 930 } } ignore_live_leader: false dest_uuid: "8d90d61da62549208fe117b4384a469b" is_pre_election: true
I20260812 06:36:27.923980  6703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" candidate_term: 9 candidate_status { last_received { term: 8 index: 844 } } ignore_live_leader: false dest_uuid: "42895a9bcba64aa484f6509605035d02" is_pre_election: true
I20260812 06:36:27.924021  6247 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7962f587ccd842ad8a3f6ed9a5d7707e in term 8.
I20260812 06:36:27.924140  6703 raft_consensus.cc:2410] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b037d17b21f8422fa5e41b70eb4dcd08 for term 9 because replica has last-logged OpId of term: 8 index: 935, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 844.
I20260812 06:36:27.924256  6479 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" candidate_term: 9 candidate_status { last_received { term: 8 index: 844 } } ignore_live_leader: false dest_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" is_pre_election: true
I20260812 06:36:27.924315  6247 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" candidate_term: 9 candidate_status { last_received { term: 8 index: 844 } } ignore_live_leader: false dest_uuid: "8d90d61da62549208fe117b4384a469b" is_pre_election: true
I20260812 06:36:27.924417  6479 raft_consensus.cc:2410] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b037d17b21f8422fa5e41b70eb4dcd08 for term 9 because replica has last-logged OpId of term: 8 index: 930, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 844.
I20260812 06:36:27.924396  6919 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" candidate_term: 9 candidate_status { last_received { term: 8 index: 844 } } ignore_live_leader: false dest_uuid: "222b20de5a1145c6b8a11df8374e8790" is_pre_election: true
I20260812 06:36:27.924458  6247 raft_consensus.cc:2410] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b037d17b21f8422fa5e41b70eb4dcd08 for term 9 because replica has last-logged OpId of term: 8 index: 880, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 844.
I20260812 06:36:27.924505  6919 raft_consensus.cc:2410] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b037d17b21f8422fa5e41b70eb4dcd08 for term 9 because replica has last-logged OpId of term: 8 index: 935, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 844.
I20260812 06:36:27.924674  6047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" candidate_term: 9 candidate_status { last_received { term: 8 index: 844 } } ignore_live_leader: false dest_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" is_pre_election: true
I20260812 06:36:27.924808  6047 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P ec6c8d818e744cdbb6b0d15b321010e0 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b037d17b21f8422fa5e41b70eb4dcd08 in term 8.
I20260812 06:36:27.924932  6369 leader_election.cc:304] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 5 responses out of 7 voters: 4 yes votes; 1 no votes. yes voters: 7962f587ccd842ad8a3f6ed9a5d7707e, 8d90d61da62549208fe117b4384a469b, b037d17b21f8422fa5e41b70eb4dcd08, ec6c8d818e744cdbb6b0d15b321010e0; no voters: 222b20de5a1145c6b8a11df8374e8790
I20260812 06:36:27.924925  7211 leader_election.cc:304] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 5 responses out of 7 voters: 1 yes votes; 4 no votes. yes voters: b037d17b21f8422fa5e41b70eb4dcd08; no voters: 222b20de5a1145c6b8a11df8374e8790, 42895a9bcba64aa484f6509605035d02, 7962f587ccd842ad8a3f6ed9a5d7707e, 8d90d61da62549208fe117b4384a469b
I20260812 06:36:27.925043  6701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" candidate_term: 9 candidate_status { last_received { term: 8 index: 930 } } ignore_live_leader: false dest_uuid: "42895a9bcba64aa484f6509605035d02" is_pre_election: true
I20260812 06:36:27.925096  8048 raft_consensus.cc:2804] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260812 06:36:27.925148  8048 raft_consensus.cc:493] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [term 8 FOLLOWER]: Starting leader election (detected failure of leader 2df4308c852240859ada53d216ab69ae)
I20260812 06:36:27.925187  6701 raft_consensus.cc:2410] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7962f587ccd842ad8a3f6ed9a5d7707e for term 9 because replica has last-logged OpId of term: 8 index: 935, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 930.
I20260812 06:36:27.925204  8048 raft_consensus.cc:3060] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [term 8 FOLLOWER]: Advancing to term 9
I20260812 06:36:27.925159  8049 raft_consensus.cc:2749] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260812 06:36:27.926134  8048 raft_consensus.cc:515] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df4308c852240859ada53d216ab69ae" member_type: VOTER last_known_addr { host: "127.4.177.6" port: 34269 } } peers { permanent_uuid: "42895a9bcba64aa484f6509605035d02" member_type: VOTER last_known_addr { host: "127.4.177.4" port: 38181 } } peers { permanent_uuid: "222b20de5a1145c6b8a11df8374e8790" member_type: VOTER last_known_addr { host: "127.4.177.5" port: 43521 } } peers { permanent_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 40897 } } peers { permanent_uuid: "8d90d61da62549208fe117b4384a469b" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 41539 } } peers { permanent_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" member_type: VOTER last_known_addr { host: "127.4.177.7" port: 46299 } } peers { permanent_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 41057 } }
I20260812 06:36:27.926719  8048 leader_election.cc:290] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [CANDIDATE]: Term 9 election: Requested vote from peers 2df4308c852240859ada53d216ab69ae (127.4.177.6:34269), 42895a9bcba64aa484f6509605035d02 (127.4.177.4:38181), 222b20de5a1145c6b8a11df8374e8790 (127.4.177.5:43521), 8d90d61da62549208fe117b4384a469b (127.4.177.2:41539), b037d17b21f8422fa5e41b70eb4dcd08 (127.4.177.7:46299), ec6c8d818e744cdbb6b0d15b321010e0 (127.4.177.1:41057)
I20260812 06:36:27.927053  7321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" candidate_term: 9 candidate_status { last_received { term: 8 index: 930 } } ignore_live_leader: false dest_uuid: "b037d17b21f8422fa5e41b70eb4dcd08"
I20260812 06:36:27.927167  7321 raft_consensus.cc:3060] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08 [term 8 FOLLOWER]: Advancing to term 9
I20260812 06:36:27.927184  6919 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" candidate_term: 9 candidate_status { last_received { term: 8 index: 930 } } ignore_live_leader: false dest_uuid: "222b20de5a1145c6b8a11df8374e8790"
I20260812 06:36:27.927210  6047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" candidate_term: 9 candidate_status { last_received { term: 8 index: 930 } } ignore_live_leader: false dest_uuid: "ec6c8d818e744cdbb6b0d15b321010e0"
I20260812 06:36:27.927281  6919 raft_consensus.cc:3060] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [term 8 FOLLOWER]: Advancing to term 9
I20260812 06:36:27.927300  6047 raft_consensus.cc:3060] T 7065bb403e3047dd9a0853c49318ef1f P ec6c8d818e744cdbb6b0d15b321010e0 [term 8 FOLLOWER]: Advancing to term 9
I20260812 06:36:27.928251  6919 raft_consensus.cc:2410] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 7962f587ccd842ad8a3f6ed9a5d7707e for term 9 because replica has last-logged OpId of term: 8 index: 935, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 930.
I20260812 06:36:27.928289  7321 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7962f587ccd842ad8a3f6ed9a5d7707e in term 9.
I20260812 06:36:27.928289  6047 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P ec6c8d818e744cdbb6b0d15b321010e0 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7962f587ccd842ad8a3f6ed9a5d7707e in term 9.
I20260812 06:36:27.928823  6247 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" candidate_term: 9 candidate_status { last_received { term: 8 index: 930 } } ignore_live_leader: false dest_uuid: "8d90d61da62549208fe117b4384a469b"
I20260812 06:36:27.928853  6701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" candidate_term: 9 candidate_status { last_received { term: 8 index: 930 } } ignore_live_leader: false dest_uuid: "42895a9bcba64aa484f6509605035d02"
I20260812 06:36:27.928939  6701 raft_consensus.cc:3060] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [term 8 FOLLOWER]: Advancing to term 9
I20260812 06:36:27.928965  6247 raft_consensus.cc:3060] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b [term 8 FOLLOWER]: Advancing to term 9
I20260812 06:36:27.929795  6701 raft_consensus.cc:2410] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 7962f587ccd842ad8a3f6ed9a5d7707e for term 9 because replica has last-logged OpId of term: 8 index: 935, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 930.
I20260812 06:36:27.929793  6247 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7962f587ccd842ad8a3f6ed9a5d7707e in term 9.
I20260812 06:36:27.930126  6369 leader_election.cc:304] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 5 responses out of 7 voters: 4 yes votes; 1 no votes. yes voters: 7962f587ccd842ad8a3f6ed9a5d7707e, 8d90d61da62549208fe117b4384a469b, b037d17b21f8422fa5e41b70eb4dcd08, ec6c8d818e744cdbb6b0d15b321010e0; no voters: 222b20de5a1145c6b8a11df8374e8790
I20260812 06:36:27.930351  8048 raft_consensus.cc:2804] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [term 9 FOLLOWER]: Leader election won for term 9
I20260812 06:36:27.930565  8048 raft_consensus.cc:697] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [term 9 LEADER]: Becoming Leader. State: Replica: 7962f587ccd842ad8a3f6ed9a5d7707e, State: Running, Role: LEADER
I20260812 06:36:27.930744  8048 consensus_queue.cc:237] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 927, Committed index: 927, Last appended: 8.930, Last appended by leader: 930, Current term: 9, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df4308c852240859ada53d216ab69ae" member_type: VOTER last_known_addr { host: "127.4.177.6" port: 34269 } } peers { permanent_uuid: "42895a9bcba64aa484f6509605035d02" member_type: VOTER last_known_addr { host: "127.4.177.4" port: 38181 } } peers { permanent_uuid: "222b20de5a1145c6b8a11df8374e8790" member_type: VOTER last_known_addr { host: "127.4.177.5" port: 43521 } } peers { permanent_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 40897 } } peers { permanent_uuid: "8d90d61da62549208fe117b4384a469b" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 41539 } } peers { permanent_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" member_type: VOTER last_known_addr { host: "127.4.177.7" port: 46299 } } peers { permanent_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 41057 } }
I20260812 06:36:27.932946  5812 catalog_manager.cc:5719] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e reported cstate change: term changed from 8 to 9, leader changed from 2df4308c852240859ada53d216ab69ae (127.4.177.6) to 7962f587ccd842ad8a3f6ed9a5d7707e (127.4.177.3). New cstate: current_term: 9 leader_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df4308c852240859ada53d216ab69ae" member_type: VOTER last_known_addr { host: "127.4.177.6" port: 34269 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "42895a9bcba64aa484f6509605035d02" member_type: VOTER last_known_addr { host: "127.4.177.4" port: 38181 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "222b20de5a1145c6b8a11df8374e8790" member_type: VOTER last_known_addr { host: "127.4.177.5" port: 43521 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 40897 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8d90d61da62549208fe117b4384a469b" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 41539 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" member_type: VOTER last_known_addr { host: "127.4.177.7" port: 46299 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 41057 } health_report { overall_health: UNKNOWN } } }
I20260812 06:36:28.268363  7108 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" candidate_term: 9 candidate_status { last_received { term: 8 index: 805 } } ignore_live_leader: false dest_uuid: "2df4308c852240859ada53d216ab69ae" is_pre_election: true
W20260812 06:36:28.268575  7017 connection.cc:256] RPC call timeout handler was delayed by -1.518797738943249s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1s for a response
W20260812 06:36:28.268859  7020 connection.cc:256] RPC call timeout handler was delayed by -1.6302853771485388s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1s for a response
W20260812 06:36:28.268921  7020 connection.cc:256] RPC call timeout handler was delayed by -1.4081006532069296s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1s for a response
W20260812 06:36:28.268968  7020 connection.cc:256] RPC call timeout handler was delayed by -1.2971610052045435s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1s for a response
I20260812 06:36:28.269546  7108 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "222b20de5a1145c6b8a11df8374e8790" candidate_term: 9 candidate_status { last_received { term: 8 index: 935 } } ignore_live_leader: false dest_uuid: "2df4308c852240859ada53d216ab69ae" is_pre_election: true
I20260812 06:36:28.270083  6701 raft_consensus.cc:1240] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [term 9 FOLLOWER]: Rejecting Update request from peer 2df4308c852240859ada53d216ab69ae for earlier term 8. Current term is 9. Ops: []
I20260812 06:36:28.270790  7109 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" candidate_term: 9 candidate_status { last_received { term: 8 index: 844 } } ignore_live_leader: false dest_uuid: "2df4308c852240859ada53d216ab69ae" is_pre_election: true
I20260812 06:36:28.271071  7107 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" candidate_term: 9 candidate_status { last_received { term: 8 index: 930 } } ignore_live_leader: false dest_uuid: "2df4308c852240859ada53d216ab69ae" is_pre_election: true
I20260812 06:36:28.271291  7106 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" candidate_term: 9 candidate_status { last_received { term: 8 index: 930 } } ignore_live_leader: false dest_uuid: "2df4308c852240859ada53d216ab69ae"
I20260812 06:36:28.274091  8061 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:50492 (ReqId={client: 3067f3aef2a44cc4bb7f2ccd6acf83db, seq_no=923, attempt_no=0}) took 2210 ms. Trace:
I20260812 06:36:28.274281  8061 rpcz_store.cc:276] 0812 06:36:26.063652 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 06:36:26.063760 (+   108us) service_pool.cc:224] Handling call
0812 06:36:28.274076 (+2210316us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0812 06:36:26.067614 (+     0us) write_op.cc:183] PREPARE: starting on tablet 7065bb403e3047dd9a0853c49318ef1f
0812 06:36:26.067686 (+    72us) write_op.cc:432] Acquiring schema lock in shared mode
0812 06:36:26.067689 (+     3us) write_op.cc:435] Acquired schema lock
0812 06:36:26.067690 (+     1us) tablet.cc:662] Decoding operations
0812 06:36:26.067778 (+    88us) write_op.cc:620] Acquiring the partition lock for write op
0812 06:36:26.067783 (+     5us) write_op.cc:641] Partition lock acquired for write op
0812 06:36:26.067786 (+     3us) tablet.cc:685] Acquiring locks for 10 operations
0812 06:36:26.067868 (+    82us) tablet.cc:701] Row locks acquired
0812 06:36:26.067869 (+     1us) write_op.cc:260] PREPARE: finished
0812 06:36:26.067898 (+    29us) write_op.cc:270] Start()
0812 06:36:26.067918 (+    20us) write_op.cc:276] Timestamp: P: 1786516586067817 usec, L: 0
0812 06:36:26.067920 (+     2us) op_driver.cc:348] REPLICATION: starting
0812 06:36:26.068028 (+   108us) log.cc:844] Serialized 696 byte log entry
0812 06:36:28.272480 (+2204452us) op_driver.cc:464] REPLICATION: finished
0812 06:36:28.273403 (+   923us) write_op.cc:301] APPLY: starting
0812 06:36:28.273428 (+    25us) tablet.cc:1367] starting BulkCheckPresence
0812 06:36:28.273470 (+    42us) tablet.cc:1370] finished BulkCheckPresence
0812 06:36:28.273483 (+    13us) tablet.cc:1372] starting ApplyRowOperation cycle
0812 06:36:28.273695 (+   212us) tablet.cc:1383] finished ApplyRowOperation cycle
0812 06:36:28.273736 (+    41us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0812 06:36:28.273742 (+     6us) write_op.cc:312] APPLY: finished
0812 06:36:28.273825 (+    83us) log.cc:844] Serialized 77 byte log entry
0812 06:36:28.273981 (+   156us) write_op.cc:489] Releasing partition, row and schema locks
0812 06:36:28.273998 (+    17us) write_op.cc:454] Released schema lock
0812 06:36:28.274005 (+     7us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":860,"mutex_wait_us":29,"num_ops":10,"prepare.queue_time_us":3601,"prepare.run_cpu_time_us":496,"prepare.run_wall_time_us":813,"raft.queue_time_us":23,"raft.run_cpu_time_us":114,"raft.run_wall_time_us":114,"replication_time_us":2204530,"thread_start_us":155,"threads_started":2}]]}
I20260812 06:36:28.275406  8043 consensus_queue.cc:1059] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "42895a9bcba64aa484f6509605035d02" member_type: VOTER last_known_addr { host: "127.4.177.4" port: 38181 }, Status: INVALID_TERM, Last received: 8.935, Next index: 936, Last known committed idx: 927, Time since last communication: 0.000s
I20260812 06:36:28.275565  6479 raft_consensus.cc:1240] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [term 9 LEADER]: Rejecting Update request from peer 2df4308c852240859ada53d216ab69ae for earlier term 8. Current term is 9. Ops: [8.931-8.935]
I20260812 06:36:28.275590  8043 raft_consensus.cc:3055] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [term 8 LEADER]: Stepping down as leader of term 8
I20260812 06:36:28.275645  8043 raft_consensus.cc:740] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 2df4308c852240859ada53d216ab69ae, State: Running, Role: LEADER
I20260812 06:36:28.275799  8043 consensus_queue.cc:260] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 930, Committed index: 930, Last appended: 8.935, Last appended by leader: 935, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df4308c852240859ada53d216ab69ae" member_type: VOTER last_known_addr { host: "127.4.177.6" port: 34269 } } peers { permanent_uuid: "42895a9bcba64aa484f6509605035d02" member_type: VOTER last_known_addr { host: "127.4.177.4" port: 38181 } } peers { permanent_uuid: "222b20de5a1145c6b8a11df8374e8790" member_type: VOTER last_known_addr { host: "127.4.177.5" port: 43521 } } peers { permanent_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 40897 } } peers { permanent_uuid: "8d90d61da62549208fe117b4384a469b" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 41539 } } peers { permanent_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" member_type: VOTER last_known_addr { host: "127.4.177.7" port: 46299 } } peers { permanent_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 41057 } }
I20260812 06:36:28.276087  8043 raft_consensus.cc:3060] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [term 8 FOLLOWER]: Advancing to term 9
I20260812 06:36:28.276785  8061 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:50492 (ReqId={client: 3067f3aef2a44cc4bb7f2ccd6acf83db, seq_no=924, attempt_no=0}) took 2213 ms. Trace:
I20260812 06:36:28.276844  8061 rpcz_store.cc:276] 0812 06:36:26.063722 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 06:36:26.064032 (+   310us) service_pool.cc:224] Handling call
0812 06:36:28.276771 (+2212739us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0812 06:36:26.068477 (+     0us) write_op.cc:183] PREPARE: starting on tablet 7065bb403e3047dd9a0853c49318ef1f
0812 06:36:26.068559 (+    82us) write_op.cc:432] Acquiring schema lock in shared mode
0812 06:36:26.068562 (+     3us) write_op.cc:435] Acquired schema lock
0812 06:36:26.068563 (+     1us) tablet.cc:662] Decoding operations
0812 06:36:26.068646 (+    83us) write_op.cc:620] Acquiring the partition lock for write op
0812 06:36:26.068654 (+     8us) write_op.cc:641] Partition lock acquired for write op
0812 06:36:26.068657 (+     3us) tablet.cc:685] Acquiring locks for 10 operations
0812 06:36:26.068728 (+    71us) tablet.cc:701] Row locks acquired
0812 06:36:26.068731 (+     3us) write_op.cc:260] PREPARE: finished
0812 06:36:26.068758 (+    27us) write_op.cc:270] Start()
0812 06:36:26.068776 (+    18us) write_op.cc:276] Timestamp: P: 1786516586068677 usec, L: 0
0812 06:36:26.068777 (+     1us) op_driver.cc:348] REPLICATION: starting
0812 06:36:26.068875 (+    98us) log.cc:844] Serialized 686 byte log entry
0812 06:36:28.272625 (+2203750us) op_driver.cc:464] REPLICATION: finished
0812 06:36:28.275583 (+  2958us) write_op.cc:301] APPLY: starting
0812 06:36:28.275599 (+    16us) tablet.cc:1367] starting BulkCheckPresence
0812 06:36:28.275632 (+    33us) tablet.cc:1370] finished BulkCheckPresence
0812 06:36:28.275643 (+    11us) tablet.cc:1372] starting ApplyRowOperation cycle
0812 06:36:28.275810 (+   167us) tablet.cc:1383] finished ApplyRowOperation cycle
0812 06:36:28.275839 (+    29us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0812 06:36:28.275843 (+     4us) write_op.cc:312] APPLY: finished
0812 06:36:28.275903 (+    60us) log.cc:844] Serialized 77 byte log entry
0812 06:36:28.275936 (+    33us) write_op.cc:489] Releasing partition, row and schema locks
0812 06:36:28.275948 (+    12us) write_op.cc:454] Released schema lock
0812 06:36:28.275953 (+     5us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":2916,"num_ops":10,"prepare.queue_time_us":4200,"prepare.run_cpu_time_us":434,"prepare.run_wall_time_us":433,"replication_time_us":2203832,"thread_start_us":60,"threads_started":1}]]}
W20260812 06:36:28.277319  7061 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50492: Illegal state: replica 2df4308c852240859ada53d216ab69ae is not leader of this config: current role FOLLOWER
I20260812 06:36:28.277558  8061 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:50492 (ReqId={client: 3067f3aef2a44cc4bb7f2ccd6acf83db, seq_no=925, attempt_no=0}) took 2213 ms. Trace:
I20260812 06:36:28.277652  8061 rpcz_store.cc:276] 0812 06:36:26.064543 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 06:36:26.064583 (+    40us) service_pool.cc:224] Handling call
0812 06:36:28.277549 (+2212966us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0812 06:36:26.068939 (+     0us) write_op.cc:183] PREPARE: starting on tablet 7065bb403e3047dd9a0853c49318ef1f
0812 06:36:26.069009 (+    70us) write_op.cc:432] Acquiring schema lock in shared mode
0812 06:36:26.069012 (+     3us) write_op.cc:435] Acquired schema lock
0812 06:36:26.069014 (+     2us) tablet.cc:662] Decoding operations
0812 06:36:26.069100 (+    86us) write_op.cc:620] Acquiring the partition lock for write op
0812 06:36:26.069106 (+     6us) write_op.cc:641] Partition lock acquired for write op
0812 06:36:26.069108 (+     2us) tablet.cc:685] Acquiring locks for 10 operations
0812 06:36:26.069169 (+    61us) tablet.cc:701] Row locks acquired
0812 06:36:26.069171 (+     2us) write_op.cc:260] PREPARE: finished
0812 06:36:26.069196 (+    25us) write_op.cc:270] Start()
0812 06:36:26.069214 (+    18us) write_op.cc:276] Timestamp: P: 1786516586069116 usec, L: 0
0812 06:36:26.069215 (+     1us) op_driver.cc:348] REPLICATION: starting
0812 06:36:26.069321 (+   106us) log.cc:844] Serialized 696 byte log entry
0812 06:36:28.272736 (+2203415us) op_driver.cc:464] REPLICATION: finished
0812 06:36:28.277172 (+  4436us) write_op.cc:301] APPLY: starting
0812 06:36:28.277182 (+    10us) tablet.cc:1367] starting BulkCheckPresence
0812 06:36:28.277209 (+    27us) tablet.cc:1370] finished BulkCheckPresence
0812 06:36:28.277220 (+    11us) tablet.cc:1372] starting ApplyRowOperation cycle
0812 06:36:28.277377 (+   157us) tablet.cc:1383] finished ApplyRowOperation cycle
0812 06:36:28.277402 (+    25us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0812 06:36:28.277407 (+     5us) write_op.cc:312] APPLY: finished
0812 06:36:28.277456 (+    49us) log.cc:844] Serialized 77 byte log entry
0812 06:36:28.277487 (+    31us) write_op.cc:489] Releasing partition, row and schema locks
0812 06:36:28.277498 (+    11us) write_op.cc:454] Released schema lock
0812 06:36:28.277502 (+     4us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":4410,"num_ops":10,"prepare.queue_time_us":4119,"prepare.run_cpu_time_us":415,"prepare.run_wall_time_us":415,"replication_time_us":2203506,"thread_start_us":44,"threads_started":1}]]}
I20260812 06:36:28.277791  6919 raft_consensus.cc:1240] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [term 9 FOLLOWER]: Rejecting Update request from peer 2df4308c852240859ada53d216ab69ae for earlier term 8. Current term is 9. Ops: []
I20260812 06:36:28.277931  6047 raft_consensus.cc:1240] T 7065bb403e3047dd9a0853c49318ef1f P ec6c8d818e744cdbb6b0d15b321010e0 [term 9 FOLLOWER]: Rejecting Update request from peer 2df4308c852240859ada53d216ab69ae for earlier term 8. Current term is 9. Ops: [8.806-8.935]
I20260812 06:36:28.279919  7321 raft_consensus.cc:1240] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08 [term 9 FOLLOWER]: Rejecting Update request from peer 2df4308c852240859ada53d216ab69ae for earlier term 8. Current term is 9. Ops: [8.845-8.935]
I20260812 06:36:28.281108  6247 raft_consensus.cc:1240] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b [term 9 FOLLOWER]: Rejecting Update request from peer 2df4308c852240859ada53d216ab69ae for earlier term 8. Current term is 9. Ops: [8.881-8.935]
W20260812 06:36:28.281849  7061 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50492: Illegal state: replica 2df4308c852240859ada53d216ab69ae is not leader of this config: current role FOLLOWER
W20260812 06:36:28.282047  7049 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50492: Illegal state: replica 2df4308c852240859ada53d216ab69ae is not leader of this config: current role FOLLOWER
W20260812 06:36:28.284971  6621 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:42408: Illegal state: replica 42895a9bcba64aa484f6509605035d02 is not leader of this config: current role FOLLOWER
W20260812 06:36:28.285085  6626 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:42408: Illegal state: replica 42895a9bcba64aa484f6509605035d02 is not leader of this config: current role FOLLOWER
W20260812 06:36:28.287228  6621 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:42408: Illegal state: replica 42895a9bcba64aa484f6509605035d02 is not leader of this config: current role FOLLOWER
W20260812 06:36:28.288379  6848 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:57072: Illegal state: replica 222b20de5a1145c6b8a11df8374e8790 is not leader of this config: current role FOLLOWER
W20260812 06:36:28.289378  6848 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:57072: Illegal state: replica 222b20de5a1145c6b8a11df8374e8790 is not leader of this config: current role FOLLOWER
W20260812 06:36:28.295534  6848 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:57072: Illegal state: replica 222b20de5a1145c6b8a11df8374e8790 is not leader of this config: current role FOLLOWER
I20260812 06:36:28.297273  7321 raft_consensus.cc:1275] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08 [term 9 FOLLOWER]: Refusing update from remote peer 7962f587ccd842ad8a3f6ed9a5d7707e: Log matching property violated. Preceding OpId in replica: term: 8 index: 844. Preceding OpId from leader: term: 9 index: 932. (index mismatch)
I20260812 06:36:28.297699  8068 consensus_queue.cc:1048] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [LEADER]: Connected to new peer: Peer: permanent_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" member_type: VOTER last_known_addr { host: "127.4.177.7" port: 46299 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 931, Last known committed idx: 837, Time since last communication: 0.000s
I20260812 06:36:28.297837  7099 raft_consensus.cc:1275] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [term 9 FOLLOWER]: Refusing update from remote peer 7962f587ccd842ad8a3f6ed9a5d7707e: Log matching property violated. Preceding OpId in replica: term: 8 index: 935. Preceding OpId from leader: term: 9 index: 932. (term mismatch)
I20260812 06:36:28.297844  6047 raft_consensus.cc:1275] T 7065bb403e3047dd9a0853c49318ef1f P ec6c8d818e744cdbb6b0d15b321010e0 [term 9 FOLLOWER]: Refusing update from remote peer 7962f587ccd842ad8a3f6ed9a5d7707e: Log matching property violated. Preceding OpId in replica: term: 8 index: 805. Preceding OpId from leader: term: 9 index: 932. (index mismatch)
I20260812 06:36:28.297911  7099 pending_rounds.cc:85] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae: Aborting all ops after (but not including) 931
I20260812 06:36:28.297935  6247 raft_consensus.cc:1275] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b [term 9 FOLLOWER]: Refusing update from remote peer 7962f587ccd842ad8a3f6ed9a5d7707e: Log matching property violated. Preceding OpId in replica: term: 8 index: 880. Preceding OpId from leader: term: 9 index: 932. (index mismatch)
I20260812 06:36:28.297993  7099 pending_rounds.cc:107] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae: Aborting uncommitted WRITE_OP operation due to leader change: 8.932
W20260812 06:36:28.298139  7099 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50492: Aborted: Op aborted by new leader
I20260812 06:36:28.298291  7099 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:50492 (ReqId={client: 3067f3aef2a44cc4bb7f2ccd6acf83db, seq_no=928, attempt_no=0}) took 2226 ms. Trace:
I20260812 06:36:28.298413  8048 consensus_queue.cc:1048] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8d90d61da62549208fe117b4384a469b" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 41539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 931, Last known committed idx: 876, Time since last communication: 0.000s
I20260812 06:36:28.298354  7099 rpcz_store.cc:276] 0812 06:36:26.071350 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 06:36:26.071388 (+    38us) service_pool.cc:224] Handling call
0812 06:36:28.298280 (+2226892us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0812 06:36:26.074421 (+     0us) write_op.cc:183] PREPARE: starting on tablet 7065bb403e3047dd9a0853c49318ef1f
0812 06:36:26.074492 (+    71us) write_op.cc:432] Acquiring schema lock in shared mode
0812 06:36:26.074495 (+     3us) write_op.cc:435] Acquired schema lock
0812 06:36:26.074497 (+     2us) tablet.cc:662] Decoding operations
0812 06:36:26.074588 (+    91us) write_op.cc:620] Acquiring the partition lock for write op
0812 06:36:26.074593 (+     5us) write_op.cc:641] Partition lock acquired for write op
0812 06:36:26.074597 (+     4us) tablet.cc:685] Acquiring locks for 10 operations
0812 06:36:26.074664 (+    67us) tablet.cc:701] Row locks acquired
0812 06:36:26.074665 (+     1us) write_op.cc:260] PREPARE: finished
0812 06:36:26.075398 (+   733us) write_op.cc:270] Start()
0812 06:36:26.075427 (+    29us) write_op.cc:276] Timestamp: P: 1786516586074611 usec, L: 0
0812 06:36:26.075429 (+     2us) op_driver.cc:348] REPLICATION: starting
0812 06:36:26.075510 (+    81us) log.cc:844] Serialized 686 byte log entry
0812 06:36:28.298091 (+2222581us) op_driver.cc:464] REPLICATION: finished
0812 06:36:28.298100 (+     9us) op_driver.cc:389] HandleFailure(Aborted: Op aborted by new leader)
0812 06:36:28.298104 (+     4us) write_op.cc:489] Releasing partition, row and schema locks
0812 06:36:28.298129 (+    25us) write_op.cc:454] Released schema lock
0812 06:36:28.298135 (+     6us) write_op.cc:335] FINISH: Op aborted
Metrics: {"child_traces":[["op",{"num_ops":10,"prepare.queue_time_us":2782,"prepare.run_cpu_time_us":467,"prepare.run_wall_time_us":1165,"replication_time_us":2222610}]]}
I20260812 06:36:28.298521  6701 raft_consensus.cc:1275] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [term 9 FOLLOWER]: Refusing update from remote peer 7962f587ccd842ad8a3f6ed9a5d7707e: Log matching property violated. Preceding OpId in replica: term: 8 index: 935. Preceding OpId from leader: term: 9 index: 932. (term mismatch)
I20260812 06:36:28.298583  6701 pending_rounds.cc:85] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02: Aborting all ops after (but not including) 931
I20260812 06:36:28.298640  7099 pending_rounds.cc:107] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae: Aborting uncommitted WRITE_OP operation due to leader change: 8.933
I20260812 06:36:28.298646  6701 pending_rounds.cc:107] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02: Aborting uncommitted WRITE_OP operation due to leader change: 8.932
W20260812 06:36:28.298763  7099 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50492: Aborted: Op aborted by new leader
I20260812 06:36:28.298856  6701 pending_rounds.cc:107] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02: Aborting uncommitted WRITE_OP operation due to leader change: 8.933
I20260812 06:36:28.298864  7099 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:50492 (ReqId={client: 3067f3aef2a44cc4bb7f2ccd6acf83db, seq_no=927, attempt_no=0}) took 2227 ms. Trace:
I20260812 06:36:28.299005  6701 pending_rounds.cc:107] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02: Aborting uncommitted WRITE_OP operation due to leader change: 8.934
I20260812 06:36:28.298900  7099 rpcz_store.cc:276] 0812 06:36:26.071278 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 06:36:26.071654 (+   376us) service_pool.cc:224] Handling call
0812 06:36:28.298855 (+2227201us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0812 06:36:26.075604 (+     0us) write_op.cc:183] PREPARE: starting on tablet 7065bb403e3047dd9a0853c49318ef1f
0812 06:36:26.075674 (+    70us) write_op.cc:432] Acquiring schema lock in shared mode
0812 06:36:26.075677 (+     3us) write_op.cc:435] Acquired schema lock
0812 06:36:26.075678 (+     1us) tablet.cc:662] Decoding operations
0812 06:36:26.075764 (+    86us) write_op.cc:620] Acquiring the partition lock for write op
0812 06:36:26.075769 (+     5us) write_op.cc:641] Partition lock acquired for write op
0812 06:36:26.075772 (+     3us) tablet.cc:685] Acquiring locks for 10 operations
0812 06:36:26.075826 (+    54us) tablet.cc:701] Row locks acquired
0812 06:36:26.075827 (+     1us) write_op.cc:260] PREPARE: finished
0812 06:36:26.075848 (+    21us) write_op.cc:270] Start()
0812 06:36:26.075882 (+    34us) write_op.cc:276] Timestamp: P: 1786516586075769 usec, L: 0
0812 06:36:26.075884 (+     2us) op_driver.cc:348] REPLICATION: starting
0812 06:36:26.075985 (+   101us) log.cc:844] Serialized 696 byte log entry
0812 06:36:28.298716 (+2222731us) op_driver.cc:464] REPLICATION: finished
0812 06:36:28.298721 (+     5us) op_driver.cc:389] HandleFailure(Aborted: Op aborted by new leader)
0812 06:36:28.298724 (+     3us) write_op.cc:489] Releasing partition, row and schema locks
0812 06:36:28.298755 (+    31us) write_op.cc:454] Released schema lock
0812 06:36:28.298760 (+     5us) write_op.cc:335] FINISH: Op aborted
Metrics: {"child_traces":[["op",{"num_ops":10,"prepare.queue_time_us":3734,"prepare.run_cpu_time_us":425,"prepare.run_wall_time_us":424,"replication_time_us":2222799}]]}
I20260812 06:36:28.299142  6701 pending_rounds.cc:107] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02: Aborting uncommitted WRITE_OP operation due to leader change: 8.935
I20260812 06:36:28.299149  7099 pending_rounds.cc:107] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae: Aborting uncommitted WRITE_OP operation due to leader change: 8.934
I20260812 06:36:28.299235  8048 consensus_queue.cc:1048] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 41057 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 931, Last known committed idx: 799, Time since last communication: 0.000s
W20260812 06:36:28.299276  7099 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50492: Aborted: Op aborted by new leader
I20260812 06:36:28.299409  7099 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:50492 (ReqId={client: 3067f3aef2a44cc4bb7f2ccd6acf83db, seq_no=929, attempt_no=0}) took 2226 ms. Trace:
I20260812 06:36:28.299458  7099 rpcz_store.cc:276] 0812 06:36:26.072496 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 06:36:26.072538 (+    42us) service_pool.cc:224] Handling call
0812 06:36:28.299400 (+2226862us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0812 06:36:26.076062 (+     0us) write_op.cc:183] PREPARE: starting on tablet 7065bb403e3047dd9a0853c49318ef1f
0812 06:36:26.076132 (+    70us) write_op.cc:432] Acquiring schema lock in shared mode
0812 06:36:26.076136 (+     4us) write_op.cc:435] Acquired schema lock
0812 06:36:26.076137 (+     1us) tablet.cc:662] Decoding operations
0812 06:36:26.076218 (+    81us) write_op.cc:620] Acquiring the partition lock for write op
0812 06:36:26.076223 (+     5us) write_op.cc:641] Partition lock acquired for write op
0812 06:36:26.076226 (+     3us) tablet.cc:685] Acquiring locks for 10 operations
0812 06:36:26.076290 (+    64us) tablet.cc:701] Row locks acquired
0812 06:36:26.076292 (+     2us) write_op.cc:260] PREPARE: finished
0812 06:36:26.076320 (+    28us) write_op.cc:270] Start()
0812 06:36:26.076338 (+    18us) write_op.cc:276] Timestamp: P: 1786516586076240 usec, L: 0
0812 06:36:26.076339 (+     1us) op_driver.cc:348] REPLICATION: starting
0812 06:36:26.076431 (+    92us) log.cc:844] Serialized 696 byte log entry
0812 06:36:28.299230 (+2222799us) op_driver.cc:464] REPLICATION: finished
0812 06:36:28.299236 (+     6us) op_driver.cc:389] HandleFailure(Aborted: Op aborted by new leader)
0812 06:36:28.299239 (+     3us) write_op.cc:489] Releasing partition, row and schema locks
0812 06:36:28.299266 (+    27us) write_op.cc:454] Released schema lock
0812 06:36:28.299272 (+     6us) write_op.cc:335] FINISH: Op aborted
Metrics: {"child_traces":[["op",{"num_ops":10,"prepare.queue_time_us":3273,"prepare.run_cpu_time_us":442,"prepare.run_wall_time_us":443,"raft.queue_time_us":32,"raft.run_cpu_time_us":20,"raft.run_wall_time_us":19,"replication_time_us":2222857,"spinlock_wait_cycles":25984}]]}
I20260812 06:36:28.299788  7099 pending_rounds.cc:107] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae: Aborting uncommitted WRITE_OP operation due to leader change: 8.935
I20260812 06:36:28.299849  6919 raft_consensus.cc:1275] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [term 9 FOLLOWER]: Refusing update from remote peer 7962f587ccd842ad8a3f6ed9a5d7707e: Log matching property violated. Preceding OpId in replica: term: 8 index: 935. Preceding OpId from leader: term: 9 index: 932. (term mismatch)
I20260812 06:36:28.299907  6919 pending_rounds.cc:85] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790: Aborting all ops after (but not including) 931
W20260812 06:36:28.299901  7099 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50492: Aborted: Op aborted by new leader
I20260812 06:36:28.299981  6919 pending_rounds.cc:107] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790: Aborting uncommitted WRITE_OP operation due to leader change: 8.932
I20260812 06:36:28.300017  7099 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:50492 (ReqId={client: 3067f3aef2a44cc4bb7f2ccd6acf83db, seq_no=930, attempt_no=0}) took 2227 ms. Trace:
I20260812 06:36:28.300168  6919 pending_rounds.cc:107] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790: Aborting uncommitted WRITE_OP operation due to leader change: 8.933
I20260812 06:36:28.300060  7099 rpcz_store.cc:276] 0812 06:36:26.073001 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 06:36:26.073039 (+    38us) service_pool.cc:224] Handling call
0812 06:36:28.300008 (+2226969us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0812 06:36:26.076527 (+     0us) write_op.cc:183] PREPARE: starting on tablet 7065bb403e3047dd9a0853c49318ef1f
0812 06:36:26.076597 (+    70us) write_op.cc:432] Acquiring schema lock in shared mode
0812 06:36:26.076600 (+     3us) write_op.cc:435] Acquired schema lock
0812 06:36:26.076602 (+     2us) tablet.cc:662] Decoding operations
0812 06:36:26.076689 (+    87us) write_op.cc:620] Acquiring the partition lock for write op
0812 06:36:26.076694 (+     5us) write_op.cc:641] Partition lock acquired for write op
0812 06:36:26.076697 (+     3us) tablet.cc:685] Acquiring locks for 10 operations
0812 06:36:26.076766 (+    69us) tablet.cc:701] Row locks acquired
0812 06:36:26.076767 (+     1us) write_op.cc:260] PREPARE: finished
0812 06:36:26.076794 (+    27us) write_op.cc:270] Start()
0812 06:36:26.076812 (+    18us) write_op.cc:276] Timestamp: P: 1786516586076713 usec, L: 0
0812 06:36:26.076815 (+     3us) op_driver.cc:348] REPLICATION: starting
0812 06:36:26.076911 (+    96us) log.cc:844] Serialized 696 byte log entry
0812 06:36:28.299864 (+2222953us) op_driver.cc:464] REPLICATION: finished
0812 06:36:28.299869 (+     5us) op_driver.cc:389] HandleFailure(Aborted: Op aborted by new leader)
0812 06:36:28.299872 (+     3us) write_op.cc:489] Releasing partition, row and schema locks
0812 06:36:28.299892 (+    20us) write_op.cc:454] Released schema lock
0812 06:36:28.299897 (+     5us) write_op.cc:335] FINISH: Op aborted
Metrics: {"child_traces":[["op",{"num_ops":10,"prepare.queue_time_us":3272,"prepare.run_cpu_time_us":448,"prepare.run_wall_time_us":448,"replication_time_us":2223015}]]}
I20260812 06:36:28.300319  6919 pending_rounds.cc:107] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790: Aborting uncommitted WRITE_OP operation due to leader change: 8.934
I20260812 06:36:28.300442  6919 pending_rounds.cc:107] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790: Aborting uncommitted WRITE_OP operation due to leader change: 8.935
I20260812 06:36:28.301437  8052 consensus_queue.cc:1048] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [LEADER]: Connected to new peer: Peer: permanent_uuid: "2df4308c852240859ada53d216ab69ae" member_type: VOTER last_known_addr { host: "127.4.177.6" port: 34269 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 931, Last known committed idx: 930, Time since last communication: 0.000s
I20260812 06:36:28.301555  8052 consensus_queue.cc:1243] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [LEADER]: Peer 2df4308c852240859ada53d216ab69ae log is divergent from this leader: its last log entry 8.931 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 930
I20260812 06:36:28.301769  8052 consensus_queue.cc:1048] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [LEADER]: Connected to new peer: Peer: permanent_uuid: "222b20de5a1145c6b8a11df8374e8790" member_type: VOTER last_known_addr { host: "127.4.177.5" port: 43521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 931, Last known committed idx: 927, Time since last communication: 0.000s
I20260812 06:36:28.301844  8052 consensus_queue.cc:1243] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [LEADER]: Peer 222b20de5a1145c6b8a11df8374e8790 log is divergent from this leader: its last log entry 8.931 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 927
I20260812 06:36:28.302047  8052 consensus_queue.cc:1048] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [LEADER]: Connected to new peer: Peer: permanent_uuid: "42895a9bcba64aa484f6509605035d02" member_type: VOTER last_known_addr { host: "127.4.177.4" port: 38181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 931, Last known committed idx: 927, Time since last communication: 0.000s
I20260812 06:36:28.302122  8052 consensus_queue.cc:1243] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [LEADER]: Peer 42895a9bcba64aa484f6509605035d02 log is divergent from this leader: its last log entry 8.931 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 927
I20260812 06:36:28.303198  6919 raft_consensus.cc:1217] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [term 9 FOLLOWER]: Deduplicated request from leader. Original: 8.927->[8.928-9.934]   Dedup: 8.930->[9.931-9.934]
I20260812 06:36:28.303314  6919 pending_rounds.cc:85] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790: Aborting all ops after (but not including) 930
I20260812 06:36:28.303345  6919 pending_rounds.cc:107] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790: Aborting uncommitted WRITE_OP operation due to leader change: 8.931
I20260812 06:36:28.304653  6701 raft_consensus.cc:1217] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [term 9 FOLLOWER]: Deduplicated request from leader. Original: 8.927->[8.928-9.934]   Dedup: 8.930->[9.931-9.934]
I20260812 06:36:28.304761  6701 pending_rounds.cc:85] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02: Aborting all ops after (but not including) 930
I20260812 06:36:28.304791  6701 pending_rounds.cc:107] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02: Aborting uncommitted WRITE_OP operation due to leader change: 8.931
I20260812 06:36:28.306290  7099 pending_rounds.cc:85] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae: Aborting all ops after (but not including) 930
I20260812 06:36:28.306401  7099 pending_rounds.cc:107] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae: Aborting uncommitted WRITE_OP operation due to leader change: 8.931
W20260812 06:36:28.306622  7099 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50492: Aborted: Op aborted by new leader
I20260812 06:36:28.306793  7099 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:50492 (ReqId={client: 3067f3aef2a44cc4bb7f2ccd6acf83db, seq_no=926, attempt_no=0}) took 2236 ms. Trace:
I20260812 06:36:28.306893  7099 rpcz_store.cc:276] 0812 06:36:26.070056 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 06:36:26.070166 (+   110us) service_pool.cc:224] Handling call
0812 06:36:28.306782 (+2236616us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0812 06:36:26.073939 (+     0us) write_op.cc:183] PREPARE: starting on tablet 7065bb403e3047dd9a0853c49318ef1f
0812 06:36:26.074011 (+    72us) write_op.cc:432] Acquiring schema lock in shared mode
0812 06:36:26.074014 (+     3us) write_op.cc:435] Acquired schema lock
0812 06:36:26.074015 (+     1us) tablet.cc:662] Decoding operations
0812 06:36:26.074098 (+    83us) write_op.cc:620] Acquiring the partition lock for write op
0812 06:36:26.074106 (+     8us) write_op.cc:641] Partition lock acquired for write op
0812 06:36:26.074110 (+     4us) tablet.cc:685] Acquiring locks for 10 operations
0812 06:36:26.074168 (+    58us) tablet.cc:701] Row locks acquired
0812 06:36:26.074170 (+     2us) write_op.cc:260] PREPARE: finished
0812 06:36:26.074196 (+    26us) write_op.cc:270] Start()
0812 06:36:26.074237 (+    41us) write_op.cc:276] Timestamp: P: 1786516586074115 usec, L: 0
0812 06:36:26.074239 (+     2us) op_driver.cc:348] REPLICATION: starting
0812 06:36:26.074337 (+    98us) log.cc:844] Serialized 696 byte log entry
0812 06:36:28.306537 (+2232200us) op_driver.cc:464] REPLICATION: finished
0812 06:36:28.306543 (+     6us) op_driver.cc:389] HandleFailure(Aborted: Op aborted by new leader)
0812 06:36:28.306589 (+    46us) write_op.cc:489] Releasing partition, row and schema locks
0812 06:36:28.306609 (+    20us) write_op.cc:454] Released schema lock
0812 06:36:28.306616 (+     7us) write_op.cc:335] FINISH: Op aborted
Metrics: {"child_traces":[["op",{"num_ops":10,"prepare.queue_time_us":3507,"prepare.run_cpu_time_us":453,"prepare.run_wall_time_us":453,"replication_time_us":2232261}]]}
I20260812 06:36:30.375011  8132 raft_consensus.cc:493] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 7962f587ccd842ad8a3f6ed9a5d7707e)
I20260812 06:36:30.375247  8132 raft_consensus.cc:515] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df4308c852240859ada53d216ab69ae" member_type: VOTER last_known_addr { host: "127.4.177.6" port: 34269 } } peers { permanent_uuid: "42895a9bcba64aa484f6509605035d02" member_type: VOTER last_known_addr { host: "127.4.177.4" port: 38181 } } peers { permanent_uuid: "222b20de5a1145c6b8a11df8374e8790" member_type: VOTER last_known_addr { host: "127.4.177.5" port: 43521 } } peers { permanent_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 40897 } } peers { permanent_uuid: "8d90d61da62549208fe117b4384a469b" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 41539 } } peers { permanent_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" member_type: VOTER last_known_addr { host: "127.4.177.7" port: 46299 } } peers { permanent_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 41057 } }
I20260812 06:36:30.375432  8133 raft_consensus.cc:493] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 7962f587ccd842ad8a3f6ed9a5d7707e)
I20260812 06:36:30.375581  8133 raft_consensus.cc:515] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df4308c852240859ada53d216ab69ae" member_type: VOTER last_known_addr { host: "127.4.177.6" port: 34269 } } peers { permanent_uuid: "42895a9bcba64aa484f6509605035d02" member_type: VOTER last_known_addr { host: "127.4.177.4" port: 38181 } } peers { permanent_uuid: "222b20de5a1145c6b8a11df8374e8790" member_type: VOTER last_known_addr { host: "127.4.177.5" port: 43521 } } peers { permanent_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 40897 } } peers { permanent_uuid: "8d90d61da62549208fe117b4384a469b" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 41539 } } peers { permanent_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" member_type: VOTER last_known_addr { host: "127.4.177.7" port: 46299 } } peers { permanent_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 41057 } }
I20260812 06:36:30.376067  8132 leader_election.cc:290] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 42895a9bcba64aa484f6509605035d02 (127.4.177.4:38181), 222b20de5a1145c6b8a11df8374e8790 (127.4.177.5:43521), 7962f587ccd842ad8a3f6ed9a5d7707e (127.4.177.3:40897), 8d90d61da62549208fe117b4384a469b (127.4.177.2:41539), b037d17b21f8422fa5e41b70eb4dcd08 (127.4.177.7:46299), ec6c8d818e744cdbb6b0d15b321010e0 (127.4.177.1:41057)
I20260812 06:36:30.376538  7321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "2df4308c852240859ada53d216ab69ae" candidate_term: 10 candidate_status { last_received { term: 9 index: 1100 } } ignore_live_leader: false dest_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" is_pre_election: true
I20260812 06:36:30.376576  6249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "2df4308c852240859ada53d216ab69ae" candidate_term: 10 candidate_status { last_received { term: 9 index: 1100 } } ignore_live_leader: false dest_uuid: "8d90d61da62549208fe117b4384a469b" is_pre_election: true
I20260812 06:36:30.376892  6919 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "2df4308c852240859ada53d216ab69ae" candidate_term: 10 candidate_status { last_received { term: 9 index: 1100 } } ignore_live_leader: false dest_uuid: "222b20de5a1145c6b8a11df8374e8790" is_pre_election: true
I20260812 06:36:30.376996  6249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "42895a9bcba64aa484f6509605035d02" candidate_term: 10 candidate_status { last_received { term: 9 index: 1100 } } ignore_live_leader: false dest_uuid: "8d90d61da62549208fe117b4384a469b" is_pre_election: true
I20260812 06:36:30.377482  6919 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "42895a9bcba64aa484f6509605035d02" candidate_term: 10 candidate_status { last_received { term: 9 index: 1100 } } ignore_live_leader: false dest_uuid: "222b20de5a1145c6b8a11df8374e8790" is_pre_election: true
I20260812 06:36:30.377488  7100 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "42895a9bcba64aa484f6509605035d02" candidate_term: 10 candidate_status { last_received { term: 9 index: 1100 } } ignore_live_leader: false dest_uuid: "2df4308c852240859ada53d216ab69ae" is_pre_election: true
I20260812 06:36:30.377627  7100 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 42895a9bcba64aa484f6509605035d02 in term 9.
I20260812 06:36:30.377782  8133 leader_election.cc:290] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 2df4308c852240859ada53d216ab69ae (127.4.177.6:34269), 222b20de5a1145c6b8a11df8374e8790 (127.4.177.5:43521), 7962f587ccd842ad8a3f6ed9a5d7707e (127.4.177.3:40897), 8d90d61da62549208fe117b4384a469b (127.4.177.2:41539), b037d17b21f8422fa5e41b70eb4dcd08 (127.4.177.7:46299), ec6c8d818e744cdbb6b0d15b321010e0 (127.4.177.1:41057)
I20260812 06:36:30.377866  7321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "42895a9bcba64aa484f6509605035d02" candidate_term: 10 candidate_status { last_received { term: 9 index: 1100 } } ignore_live_leader: false dest_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" is_pre_election: true
I20260812 06:36:30.377879  6047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "42895a9bcba64aa484f6509605035d02" candidate_term: 10 candidate_status { last_received { term: 9 index: 1100 } } ignore_live_leader: false dest_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" is_pre_election: true
I20260812 06:36:30.378137  6046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "2df4308c852240859ada53d216ab69ae" candidate_term: 10 candidate_status { last_received { term: 9 index: 1100 } } ignore_live_leader: false dest_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" is_pre_election: true
I20260812 06:36:30.378404  6703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "2df4308c852240859ada53d216ab69ae" candidate_term: 10 candidate_status { last_received { term: 9 index: 1100 } } ignore_live_leader: false dest_uuid: "42895a9bcba64aa484f6509605035d02" is_pre_election: true
I20260812 06:36:30.378496  7020 leader_election.cc:304] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 5 responses out of 7 voters: 1 yes votes; 4 no votes. yes voters: 2df4308c852240859ada53d216ab69ae; no voters: 222b20de5a1145c6b8a11df8374e8790, 8d90d61da62549208fe117b4384a469b, b037d17b21f8422fa5e41b70eb4dcd08, ec6c8d818e744cdbb6b0d15b321010e0
I20260812 06:36:30.378537  6703 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2df4308c852240859ada53d216ab69ae in term 9.
I20260812 06:36:30.378662  6597 leader_election.cc:304] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 6 responses out of 7 voters: 2 yes votes; 4 no votes. yes voters: 2df4308c852240859ada53d216ab69ae, 42895a9bcba64aa484f6509605035d02; no voters: 222b20de5a1145c6b8a11df8374e8790, 8d90d61da62549208fe117b4384a469b, b037d17b21f8422fa5e41b70eb4dcd08, ec6c8d818e744cdbb6b0d15b321010e0
I20260812 06:36:30.378702  8132 raft_consensus.cc:2749] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260812 06:36:30.378839  8133 raft_consensus.cc:2749] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260812 06:36:30.715416  8134 raft_consensus.cc:493] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 7962f587ccd842ad8a3f6ed9a5d7707e)
I20260812 06:36:30.715579  8134 raft_consensus.cc:515] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df4308c852240859ada53d216ab69ae" member_type: VOTER last_known_addr { host: "127.4.177.6" port: 34269 } } peers { permanent_uuid: "42895a9bcba64aa484f6509605035d02" member_type: VOTER last_known_addr { host: "127.4.177.4" port: 38181 } } peers { permanent_uuid: "222b20de5a1145c6b8a11df8374e8790" member_type: VOTER last_known_addr { host: "127.4.177.5" port: 43521 } } peers { permanent_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 40897 } } peers { permanent_uuid: "8d90d61da62549208fe117b4384a469b" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 41539 } } peers { permanent_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" member_type: VOTER last_known_addr { host: "127.4.177.7" port: 46299 } } peers { permanent_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 41057 } }
I20260812 06:36:30.716053  8135 raft_consensus.cc:493] T 7065bb403e3047dd9a0853c49318ef1f P ec6c8d818e744cdbb6b0d15b321010e0 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 7962f587ccd842ad8a3f6ed9a5d7707e)
I20260812 06:36:30.716173  8135 raft_consensus.cc:515] T 7065bb403e3047dd9a0853c49318ef1f P ec6c8d818e744cdbb6b0d15b321010e0 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df4308c852240859ada53d216ab69ae" member_type: VOTER last_known_addr { host: "127.4.177.6" port: 34269 } } peers { permanent_uuid: "42895a9bcba64aa484f6509605035d02" member_type: VOTER last_known_addr { host: "127.4.177.4" port: 38181 } } peers { permanent_uuid: "222b20de5a1145c6b8a11df8374e8790" member_type: VOTER last_known_addr { host: "127.4.177.5" port: 43521 } } peers { permanent_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 40897 } } peers { permanent_uuid: "8d90d61da62549208fe117b4384a469b" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 41539 } } peers { permanent_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" member_type: VOTER last_known_addr { host: "127.4.177.7" port: 46299 } } peers { permanent_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 41057 } }
I20260812 06:36:30.716480  8134 leader_election.cc:290] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 2df4308c852240859ada53d216ab69ae (127.4.177.6:34269), 42895a9bcba64aa484f6509605035d02 (127.4.177.4:38181), 222b20de5a1145c6b8a11df8374e8790 (127.4.177.5:43521), 7962f587ccd842ad8a3f6ed9a5d7707e (127.4.177.3:40897), 8d90d61da62549208fe117b4384a469b (127.4.177.2:41539), ec6c8d818e744cdbb6b0d15b321010e0 (127.4.177.1:41057)
I20260812 06:36:30.716959  8135 leader_election.cc:290] T 7065bb403e3047dd9a0853c49318ef1f P ec6c8d818e744cdbb6b0d15b321010e0 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 2df4308c852240859ada53d216ab69ae (127.4.177.6:34269), 42895a9bcba64aa484f6509605035d02 (127.4.177.4:38181), 222b20de5a1145c6b8a11df8374e8790 (127.4.177.5:43521), 7962f587ccd842ad8a3f6ed9a5d7707e (127.4.177.3:40897), 8d90d61da62549208fe117b4384a469b (127.4.177.2:41539), b037d17b21f8422fa5e41b70eb4dcd08 (127.4.177.7:46299)
I20260812 06:36:30.717034  6046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" candidate_term: 10 candidate_status { last_received { term: 9 index: 1014 } } ignore_live_leader: false dest_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" is_pre_election: true
I20260812 06:36:30.717089  6703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" candidate_term: 10 candidate_status { last_received { term: 9 index: 1014 } } ignore_live_leader: false dest_uuid: "42895a9bcba64aa484f6509605035d02" is_pre_election: true
I20260812 06:36:30.717160  6046 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P ec6c8d818e744cdbb6b0d15b321010e0 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b037d17b21f8422fa5e41b70eb4dcd08 in term 9.
I20260812 06:36:30.717283  6703 raft_consensus.cc:2410] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b037d17b21f8422fa5e41b70eb4dcd08 for term 10 because replica has last-logged OpId of term: 9 index: 1100, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 1014.
I20260812 06:36:30.717422  6249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" candidate_term: 10 candidate_status { last_received { term: 9 index: 1014 } } ignore_live_leader: false dest_uuid: "8d90d61da62549208fe117b4384a469b" is_pre_election: true
I20260812 06:36:30.717810  6249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" candidate_term: 10 candidate_status { last_received { term: 9 index: 980 } } ignore_live_leader: false dest_uuid: "8d90d61da62549208fe117b4384a469b" is_pre_election: true
I20260812 06:36:30.717866  7100 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" candidate_term: 10 candidate_status { last_received { term: 9 index: 1014 } } ignore_live_leader: false dest_uuid: "2df4308c852240859ada53d216ab69ae" is_pre_election: true
I20260812 06:36:30.717908  6919 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" candidate_term: 10 candidate_status { last_received { term: 9 index: 980 } } ignore_live_leader: false dest_uuid: "222b20de5a1145c6b8a11df8374e8790" is_pre_election: true
I20260812 06:36:30.718048  7100 raft_consensus.cc:2410] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b037d17b21f8422fa5e41b70eb4dcd08 for term 10 because replica has last-logged OpId of term: 9 index: 1100, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 1014.
I20260812 06:36:30.718127  6921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" candidate_term: 10 candidate_status { last_received { term: 9 index: 1014 } } ignore_live_leader: false dest_uuid: "222b20de5a1145c6b8a11df8374e8790" is_pre_election: true
I20260812 06:36:30.717880  7099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" candidate_term: 10 candidate_status { last_received { term: 9 index: 980 } } ignore_live_leader: false dest_uuid: "2df4308c852240859ada53d216ab69ae" is_pre_election: true
I20260812 06:36:30.718554  7099 raft_consensus.cc:2410] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ec6c8d818e744cdbb6b0d15b321010e0 for term 10 because replica has last-logged OpId of term: 9 index: 1100, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 980.
I20260812 06:36:30.718617  7321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" candidate_term: 10 candidate_status { last_received { term: 9 index: 980 } } ignore_live_leader: false dest_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" is_pre_election: true
I20260812 06:36:30.718633  6703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" candidate_term: 10 candidate_status { last_received { term: 9 index: 980 } } ignore_live_leader: false dest_uuid: "42895a9bcba64aa484f6509605035d02" is_pre_election: true
I20260812 06:36:30.718772  7321 raft_consensus.cc:2410] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ec6c8d818e744cdbb6b0d15b321010e0 for term 10 because replica has last-logged OpId of term: 9 index: 1014, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 980.
I20260812 06:36:30.718778  6703 raft_consensus.cc:2410] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ec6c8d818e744cdbb6b0d15b321010e0 for term 10 because replica has last-logged OpId of term: 9 index: 1100, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 980.
I20260812 06:36:30.718885  7211 leader_election.cc:304] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 6 responses out of 7 voters: 2 yes votes; 4 no votes. yes voters: b037d17b21f8422fa5e41b70eb4dcd08, ec6c8d818e744cdbb6b0d15b321010e0; no voters: 222b20de5a1145c6b8a11df8374e8790, 2df4308c852240859ada53d216ab69ae, 42895a9bcba64aa484f6509605035d02, 8d90d61da62549208fe117b4384a469b
I20260812 06:36:30.719107  8134 raft_consensus.cc:2749] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260812 06:36:30.719231  5940 leader_election.cc:304] T 7065bb403e3047dd9a0853c49318ef1f P ec6c8d818e744cdbb6b0d15b321010e0 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 5 responses out of 7 voters: 1 yes votes; 4 no votes. yes voters: ec6c8d818e744cdbb6b0d15b321010e0; no voters: 222b20de5a1145c6b8a11df8374e8790, 2df4308c852240859ada53d216ab69ae, 8d90d61da62549208fe117b4384a469b, b037d17b21f8422fa5e41b70eb4dcd08
I20260812 06:36:30.719448  8135 raft_consensus.cc:2749] T 7065bb403e3047dd9a0853c49318ef1f P ec6c8d818e744cdbb6b0d15b321010e0 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260812 06:36:30.719646  8137 raft_consensus.cc:493] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 7962f587ccd842ad8a3f6ed9a5d7707e)
I20260812 06:36:30.719760  8137 raft_consensus.cc:515] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df4308c852240859ada53d216ab69ae" member_type: VOTER last_known_addr { host: "127.4.177.6" port: 34269 } } peers { permanent_uuid: "42895a9bcba64aa484f6509605035d02" member_type: VOTER last_known_addr { host: "127.4.177.4" port: 38181 } } peers { permanent_uuid: "222b20de5a1145c6b8a11df8374e8790" member_type: VOTER last_known_addr { host: "127.4.177.5" port: 43521 } } peers { permanent_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 40897 } } peers { permanent_uuid: "8d90d61da62549208fe117b4384a469b" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 41539 } } peers { permanent_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" member_type: VOTER last_known_addr { host: "127.4.177.7" port: 46299 } } peers { permanent_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 41057 } }
I20260812 06:36:30.720467  8137 leader_election.cc:290] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 2df4308c852240859ada53d216ab69ae (127.4.177.6:34269), 42895a9bcba64aa484f6509605035d02 (127.4.177.4:38181), 222b20de5a1145c6b8a11df8374e8790 (127.4.177.5:43521), 7962f587ccd842ad8a3f6ed9a5d7707e (127.4.177.3:40897), b037d17b21f8422fa5e41b70eb4dcd08 (127.4.177.7:46299), ec6c8d818e744cdbb6b0d15b321010e0 (127.4.177.1:41057)
I20260812 06:36:30.720697  7099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "8d90d61da62549208fe117b4384a469b" candidate_term: 10 candidate_status { last_received { term: 9 index: 1046 } } ignore_live_leader: false dest_uuid: "2df4308c852240859ada53d216ab69ae" is_pre_election: true
I20260812 06:36:30.720768  7321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "8d90d61da62549208fe117b4384a469b" candidate_term: 10 candidate_status { last_received { term: 9 index: 1046 } } ignore_live_leader: false dest_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" is_pre_election: true
I20260812 06:36:30.720840  7099 raft_consensus.cc:2410] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8d90d61da62549208fe117b4384a469b for term 10 because replica has last-logged OpId of term: 9 index: 1100, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 1046.
I20260812 06:36:30.720870  7321 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8d90d61da62549208fe117b4384a469b in term 9.
I20260812 06:36:30.721079  6046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "8d90d61da62549208fe117b4384a469b" candidate_term: 10 candidate_status { last_received { term: 9 index: 1046 } } ignore_live_leader: false dest_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" is_pre_election: true
I20260812 06:36:30.721110  8141 raft_consensus.cc:493] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 7962f587ccd842ad8a3f6ed9a5d7707e)
I20260812 06:36:30.721140  6703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "8d90d61da62549208fe117b4384a469b" candidate_term: 10 candidate_status { last_received { term: 9 index: 1046 } } ignore_live_leader: false dest_uuid: "42895a9bcba64aa484f6509605035d02" is_pre_election: true
I20260812 06:36:30.721194  6046 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P ec6c8d818e744cdbb6b0d15b321010e0 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8d90d61da62549208fe117b4384a469b in term 9.
I20260812 06:36:30.721210  8141 raft_consensus.cc:515] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df4308c852240859ada53d216ab69ae" member_type: VOTER last_known_addr { host: "127.4.177.6" port: 34269 } } peers { permanent_uuid: "42895a9bcba64aa484f6509605035d02" member_type: VOTER last_known_addr { host: "127.4.177.4" port: 38181 } } peers { permanent_uuid: "222b20de5a1145c6b8a11df8374e8790" member_type: VOTER last_known_addr { host: "127.4.177.5" port: 43521 } } peers { permanent_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 40897 } } peers { permanent_uuid: "8d90d61da62549208fe117b4384a469b" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 41539 } } peers { permanent_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" member_type: VOTER last_known_addr { host: "127.4.177.7" port: 46299 } } peers { permanent_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 41057 } }
I20260812 06:36:30.721279  6703 raft_consensus.cc:2410] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8d90d61da62549208fe117b4384a469b for term 10 because replica has last-logged OpId of term: 9 index: 1100, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 1046.
I20260812 06:36:30.721740  6921 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "8d90d61da62549208fe117b4384a469b" candidate_term: 10 candidate_status { last_received { term: 9 index: 1046 } } ignore_live_leader: false dest_uuid: "222b20de5a1145c6b8a11df8374e8790" is_pre_election: true
I20260812 06:36:30.721877  6921 raft_consensus.cc:2410] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8d90d61da62549208fe117b4384a469b for term 10 because replica has last-logged OpId of term: 9 index: 1093, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 1046.
I20260812 06:36:30.721922  8141 leader_election.cc:290] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 2df4308c852240859ada53d216ab69ae (127.4.177.6:34269), 42895a9bcba64aa484f6509605035d02 (127.4.177.4:38181), 7962f587ccd842ad8a3f6ed9a5d7707e (127.4.177.3:40897), 8d90d61da62549208fe117b4384a469b (127.4.177.2:41539), b037d17b21f8422fa5e41b70eb4dcd08 (127.4.177.7:46299), ec6c8d818e744cdbb6b0d15b321010e0 (127.4.177.1:41057)
I20260812 06:36:30.722084  7099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "222b20de5a1145c6b8a11df8374e8790" candidate_term: 10 candidate_status { last_received { term: 9 index: 1093 } } ignore_live_leader: false dest_uuid: "2df4308c852240859ada53d216ab69ae" is_pre_election: true
I20260812 06:36:30.722258  7099 raft_consensus.cc:2410] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 222b20de5a1145c6b8a11df8374e8790 for term 10 because replica has last-logged OpId of term: 9 index: 1100, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 1093.
I20260812 06:36:30.722391  6046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "222b20de5a1145c6b8a11df8374e8790" candidate_term: 10 candidate_status { last_received { term: 9 index: 1093 } } ignore_live_leader: false dest_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" is_pre_election: true
I20260812 06:36:30.722517  6046 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P ec6c8d818e744cdbb6b0d15b321010e0 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 222b20de5a1145c6b8a11df8374e8790 in term 9.
I20260812 06:36:30.722579  7321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "222b20de5a1145c6b8a11df8374e8790" candidate_term: 10 candidate_status { last_received { term: 9 index: 1093 } } ignore_live_leader: false dest_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" is_pre_election: true
I20260812 06:36:30.722687  7321 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 222b20de5a1145c6b8a11df8374e8790 in term 9.
I20260812 06:36:30.722671  6249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "222b20de5a1145c6b8a11df8374e8790" candidate_term: 10 candidate_status { last_received { term: 9 index: 1093 } } ignore_live_leader: false dest_uuid: "8d90d61da62549208fe117b4384a469b" is_pre_election: true
I20260812 06:36:30.722682  6703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "222b20de5a1145c6b8a11df8374e8790" candidate_term: 10 candidate_status { last_received { term: 9 index: 1093 } } ignore_live_leader: false dest_uuid: "42895a9bcba64aa484f6509605035d02" is_pre_election: true
I20260812 06:36:30.722782  6249 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 222b20de5a1145c6b8a11df8374e8790 in term 9.
I20260812 06:36:30.722802  6703 raft_consensus.cc:2410] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 222b20de5a1145c6b8a11df8374e8790 for term 10 because replica has last-logged OpId of term: 9 index: 1100, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 1093.
I20260812 06:36:30.723009  6821 leader_election.cc:304] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 5 responses out of 7 voters: 4 yes votes; 1 no votes. yes voters: 222b20de5a1145c6b8a11df8374e8790, 8d90d61da62549208fe117b4384a469b, b037d17b21f8422fa5e41b70eb4dcd08, ec6c8d818e744cdbb6b0d15b321010e0; no voters: 2df4308c852240859ada53d216ab69ae
I20260812 06:36:30.723201  8141 raft_consensus.cc:2804] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260812 06:36:30.723256  8141 raft_consensus.cc:493] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 7962f587ccd842ad8a3f6ed9a5d7707e)
I20260812 06:36:30.723284  8141 raft_consensus.cc:3060] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [term 9 FOLLOWER]: Advancing to term 10
I20260812 06:36:30.724196  8141 raft_consensus.cc:515] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df4308c852240859ada53d216ab69ae" member_type: VOTER last_known_addr { host: "127.4.177.6" port: 34269 } } peers { permanent_uuid: "42895a9bcba64aa484f6509605035d02" member_type: VOTER last_known_addr { host: "127.4.177.4" port: 38181 } } peers { permanent_uuid: "222b20de5a1145c6b8a11df8374e8790" member_type: VOTER last_known_addr { host: "127.4.177.5" port: 43521 } } peers { permanent_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 40897 } } peers { permanent_uuid: "8d90d61da62549208fe117b4384a469b" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 41539 } } peers { permanent_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" member_type: VOTER last_known_addr { host: "127.4.177.7" port: 46299 } } peers { permanent_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 41057 } }
I20260812 06:36:30.724768  8141 leader_election.cc:290] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [CANDIDATE]: Term 10 election: Requested vote from peers 2df4308c852240859ada53d216ab69ae (127.4.177.6:34269), 42895a9bcba64aa484f6509605035d02 (127.4.177.4:38181), 7962f587ccd842ad8a3f6ed9a5d7707e (127.4.177.3:40897), 8d90d61da62549208fe117b4384a469b (127.4.177.2:41539), b037d17b21f8422fa5e41b70eb4dcd08 (127.4.177.7:46299), ec6c8d818e744cdbb6b0d15b321010e0 (127.4.177.1:41057)
I20260812 06:36:30.724906  7099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "222b20de5a1145c6b8a11df8374e8790" candidate_term: 10 candidate_status { last_received { term: 9 index: 1093 } } ignore_live_leader: false dest_uuid: "2df4308c852240859ada53d216ab69ae"
I20260812 06:36:30.725031  7099 raft_consensus.cc:3060] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [term 9 FOLLOWER]: Advancing to term 10
I20260812 06:36:30.725157  6046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "222b20de5a1145c6b8a11df8374e8790" candidate_term: 10 candidate_status { last_received { term: 9 index: 1093 } } ignore_live_leader: false dest_uuid: "ec6c8d818e744cdbb6b0d15b321010e0"
I20260812 06:36:30.725159  7321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "222b20de5a1145c6b8a11df8374e8790" candidate_term: 10 candidate_status { last_received { term: 9 index: 1093 } } ignore_live_leader: false dest_uuid: "b037d17b21f8422fa5e41b70eb4dcd08"
I20260812 06:36:30.725183  6249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "222b20de5a1145c6b8a11df8374e8790" candidate_term: 10 candidate_status { last_received { term: 9 index: 1093 } } ignore_live_leader: false dest_uuid: "8d90d61da62549208fe117b4384a469b"
I20260812 06:36:30.725255  6046 raft_consensus.cc:3060] T 7065bb403e3047dd9a0853c49318ef1f P ec6c8d818e744cdbb6b0d15b321010e0 [term 9 FOLLOWER]: Advancing to term 10
I20260812 06:36:30.725255  7321 raft_consensus.cc:3060] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08 [term 9 FOLLOWER]: Advancing to term 10
I20260812 06:36:30.725288  6249 raft_consensus.cc:3060] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b [term 9 FOLLOWER]: Advancing to term 10
I20260812 06:36:30.726269  6046 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P ec6c8d818e744cdbb6b0d15b321010e0 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 222b20de5a1145c6b8a11df8374e8790 in term 10.
I20260812 06:36:30.726269  7321 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 222b20de5a1145c6b8a11df8374e8790 in term 10.
I20260812 06:36:30.726395  6249 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 222b20de5a1145c6b8a11df8374e8790 in term 10.
I20260812 06:36:30.726678  7099 raft_consensus.cc:2410] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 222b20de5a1145c6b8a11df8374e8790 for term 10 because replica has last-logged OpId of term: 9 index: 1100, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 1093.
I20260812 06:36:30.726701  6821 leader_election.cc:304] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 7 voters: 4 yes votes; 0 no votes. yes voters: 222b20de5a1145c6b8a11df8374e8790, 8d90d61da62549208fe117b4384a469b, b037d17b21f8422fa5e41b70eb4dcd08, ec6c8d818e744cdbb6b0d15b321010e0; no voters: 
I20260812 06:36:30.726874  8141 raft_consensus.cc:2804] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [term 10 FOLLOWER]: Leader election won for term 10
I20260812 06:36:30.726899  6703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "222b20de5a1145c6b8a11df8374e8790" candidate_term: 10 candidate_status { last_received { term: 9 index: 1093 } } ignore_live_leader: false dest_uuid: "42895a9bcba64aa484f6509605035d02"
I20260812 06:36:30.727026  6703 raft_consensus.cc:3060] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [term 9 FOLLOWER]: Advancing to term 10
I20260812 06:36:30.727087  8141 raft_consensus.cc:697] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [term 10 LEADER]: Becoming Leader. State: Replica: 222b20de5a1145c6b8a11df8374e8790, State: Running, Role: LEADER
I20260812 06:36:30.727260  8141 consensus_queue.cc:237] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1091, Committed index: 1091, Last appended: 9.1093, Last appended by leader: 1093, Current term: 10, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df4308c852240859ada53d216ab69ae" member_type: VOTER last_known_addr { host: "127.4.177.6" port: 34269 } } peers { permanent_uuid: "42895a9bcba64aa484f6509605035d02" member_type: VOTER last_known_addr { host: "127.4.177.4" port: 38181 } } peers { permanent_uuid: "222b20de5a1145c6b8a11df8374e8790" member_type: VOTER last_known_addr { host: "127.4.177.5" port: 43521 } } peers { permanent_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 40897 } } peers { permanent_uuid: "8d90d61da62549208fe117b4384a469b" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 41539 } } peers { permanent_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" member_type: VOTER last_known_addr { host: "127.4.177.7" port: 46299 } } peers { permanent_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 41057 } }
I20260812 06:36:30.727967  6703 raft_consensus.cc:2410] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 222b20de5a1145c6b8a11df8374e8790 for term 10 because replica has last-logged OpId of term: 9 index: 1100, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 1093.
I20260812 06:36:30.729545  5810 catalog_manager.cc:5719] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 reported cstate change: term changed from 9 to 10, leader changed from 7962f587ccd842ad8a3f6ed9a5d7707e (127.4.177.3) to 222b20de5a1145c6b8a11df8374e8790 (127.4.177.5). New cstate: current_term: 10 leader_uuid: "222b20de5a1145c6b8a11df8374e8790" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df4308c852240859ada53d216ab69ae" member_type: VOTER last_known_addr { host: "127.4.177.6" port: 34269 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "42895a9bcba64aa484f6509605035d02" member_type: VOTER last_known_addr { host: "127.4.177.4" port: 38181 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "222b20de5a1145c6b8a11df8374e8790" member_type: VOTER last_known_addr { host: "127.4.177.5" port: 43521 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 40897 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8d90d61da62549208fe117b4384a469b" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 41539 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" member_type: VOTER last_known_addr { host: "127.4.177.7" port: 46299 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 41057 } health_report { overall_health: UNKNOWN } } }
W20260812 06:36:31.066442  6364 connection.cc:256] RPC call timeout handler was delayed by -1.52288709115237s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1s for a response
I20260812 06:36:31.066597  6480 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "222b20de5a1145c6b8a11df8374e8790" candidate_term: 10 candidate_status { last_received { term: 9 index: 1093 } } ignore_live_leader: false dest_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e"
I20260812 06:36:31.066835  6472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "8d90d61da62549208fe117b4384a469b" candidate_term: 10 candidate_status { last_received { term: 9 index: 1046 } } ignore_live_leader: false dest_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" is_pre_election: true
I20260812 06:36:31.067138  6473 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "42895a9bcba64aa484f6509605035d02" candidate_term: 10 candidate_status { last_received { term: 9 index: 1100 } } ignore_live_leader: false dest_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" is_pre_election: true
I20260812 06:36:31.067752  6474 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" candidate_term: 10 candidate_status { last_received { term: 9 index: 980 } } ignore_live_leader: false dest_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" is_pre_election: true
I20260812 06:36:31.067991  6475 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" candidate_term: 10 candidate_status { last_received { term: 9 index: 1014 } } ignore_live_leader: false dest_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" is_pre_election: true
I20260812 06:36:31.068199  6476 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "2df4308c852240859ada53d216ab69ae" candidate_term: 10 candidate_status { last_received { term: 9 index: 1100 } } ignore_live_leader: false dest_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" is_pre_election: true
I20260812 06:36:31.068384  6479 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "222b20de5a1145c6b8a11df8374e8790" candidate_term: 10 candidate_status { last_received { term: 9 index: 1093 } } ignore_live_leader: false dest_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" is_pre_election: true
W20260812 06:36:31.067158  6369 connection.cc:256] RPC call timeout handler was delayed by -1.6213253911118954s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1s for a response
W20260812 06:36:31.069052  6369 connection.cc:256] RPC call timeout handler was delayed by -1.4089627801440656s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1s for a response
W20260812 06:36:31.069098  6369 connection.cc:256] RPC call timeout handler was delayed by -1.2997527001425624s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1s for a response
I20260812 06:36:31.069686  7099 raft_consensus.cc:1240] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [term 10 FOLLOWER]: Rejecting Update request from peer 7962f587ccd842ad8a3f6ed9a5d7707e for earlier term 9. Current term is 10. Ops: []
I20260812 06:36:31.070585  6703 raft_consensus.cc:1240] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [term 10 FOLLOWER]: Rejecting Update request from peer 7962f587ccd842ad8a3f6ed9a5d7707e for earlier term 9. Current term is 10. Ops: []
I20260812 06:36:31.071265  7321 raft_consensus.cc:1240] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08 [term 10 FOLLOWER]: Rejecting Update request from peer 7962f587ccd842ad8a3f6ed9a5d7707e for earlier term 9. Current term is 10. Ops: [9.1015-9.1100]
I20260812 06:36:31.071895  8069 consensus_queue.cc:1059] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "42895a9bcba64aa484f6509605035d02" member_type: VOTER last_known_addr { host: "127.4.177.4" port: 38181 }, Status: INVALID_TERM, Last received: 9.1100, Next index: 1101, Last known committed idx: 1092, Time since last communication: 0.000s
I20260812 06:36:31.072042  8069 consensus_queue.cc:1059] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2df4308c852240859ada53d216ab69ae" member_type: VOTER last_known_addr { host: "127.4.177.6" port: 34269 }, Status: INVALID_TERM, Last received: 9.1100, Next index: 1101, Last known committed idx: 1092, Time since last communication: 0.000s
I20260812 06:36:31.072136  8069 raft_consensus.cc:3055] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [term 9 LEADER]: Stepping down as leader of term 9
I20260812 06:36:31.072173  8069 raft_consensus.cc:740] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 7962f587ccd842ad8a3f6ed9a5d7707e, State: Running, Role: LEADER
I20260812 06:36:31.072307  8069 consensus_queue.cc:260] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1093, Committed index: 1093, Last appended: 9.1100, Last appended by leader: 1100, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df4308c852240859ada53d216ab69ae" member_type: VOTER last_known_addr { host: "127.4.177.6" port: 34269 } } peers { permanent_uuid: "42895a9bcba64aa484f6509605035d02" member_type: VOTER last_known_addr { host: "127.4.177.4" port: 38181 } } peers { permanent_uuid: "222b20de5a1145c6b8a11df8374e8790" member_type: VOTER last_known_addr { host: "127.4.177.5" port: 43521 } } peers { permanent_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 40897 } } peers { permanent_uuid: "8d90d61da62549208fe117b4384a469b" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 41539 } } peers { permanent_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" member_type: VOTER last_known_addr { host: "127.4.177.7" port: 46299 } } peers { permanent_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 41057 } }
I20260812 06:36:31.072494  8069 raft_consensus.cc:3060] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [term 9 FOLLOWER]: Advancing to term 10
I20260812 06:36:31.073366  6249 raft_consensus.cc:1240] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b [term 10 FOLLOWER]: Rejecting Update request from peer 7962f587ccd842ad8a3f6ed9a5d7707e for earlier term 9. Current term is 10. Ops: [9.1047-9.1100]
I20260812 06:36:31.073764  6150 leader_election.cc:304] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 7 responses out of 7 voters: 3 yes votes; 4 no votes. yes voters: 8d90d61da62549208fe117b4384a469b, b037d17b21f8422fa5e41b70eb4dcd08, ec6c8d818e744cdbb6b0d15b321010e0; no voters: 222b20de5a1145c6b8a11df8374e8790, 2df4308c852240859ada53d216ab69ae, 42895a9bcba64aa484f6509605035d02, 7962f587ccd842ad8a3f6ed9a5d7707e
I20260812 06:36:31.074720  6046 raft_consensus.cc:1240] T 7065bb403e3047dd9a0853c49318ef1f P ec6c8d818e744cdbb6b0d15b321010e0 [term 10 FOLLOWER]: Rejecting Update request from peer 7962f587ccd842ad8a3f6ed9a5d7707e for earlier term 9. Current term is 10. Ops: [9.981-9.1100]
I20260812 06:36:31.074776  8137 raft_consensus.cc:2749] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260812 06:36:31.075445  6921 raft_consensus.cc:1240] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [term 10 LEADER]: Rejecting Update request from peer 7962f587ccd842ad8a3f6ed9a5d7707e for earlier term 9. Current term is 10. Ops: [9.1094-9.1100]
I20260812 06:36:31.076004  8154 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:33412 (ReqId={client: 3067f3aef2a44cc4bb7f2ccd6acf83db, seq_no=1087, attempt_no=0}) took 2215 ms. Trace:
I20260812 06:36:31.076086  8154 rpcz_store.cc:276] 0812 06:36:28.860218 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 06:36:28.860763 (+   545us) service_pool.cc:224] Handling call
0812 06:36:31.075981 (+2215218us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0812 06:36:28.863392 (+     0us) write_op.cc:183] PREPARE: starting on tablet 7065bb403e3047dd9a0853c49318ef1f
0812 06:36:28.863492 (+   100us) write_op.cc:432] Acquiring schema lock in shared mode
0812 06:36:28.863497 (+     5us) write_op.cc:435] Acquired schema lock
0812 06:36:28.863498 (+     1us) tablet.cc:662] Decoding operations
0812 06:36:28.863583 (+    85us) write_op.cc:620] Acquiring the partition lock for write op
0812 06:36:28.863592 (+     9us) write_op.cc:641] Partition lock acquired for write op
0812 06:36:28.863597 (+     5us) tablet.cc:685] Acquiring locks for 10 operations
0812 06:36:28.863661 (+    64us) tablet.cc:701] Row locks acquired
0812 06:36:28.863663 (+     2us) write_op.cc:260] PREPARE: finished
0812 06:36:28.863692 (+    29us) write_op.cc:270] Start()
0812 06:36:28.863710 (+    18us) write_op.cc:276] Timestamp: P: 1786516588863391 usec, L: 0
0812 06:36:28.863712 (+     2us) op_driver.cc:348] REPLICATION: starting
0812 06:36:28.865678 (+  1966us) log.cc:844] Serialized 696 byte log entry
0812 06:36:31.070916 (+2205238us) op_driver.cc:464] REPLICATION: finished
0812 06:36:31.075307 (+  4391us) write_op.cc:301] APPLY: starting
0812 06:36:31.075327 (+    20us) tablet.cc:1367] starting BulkCheckPresence
0812 06:36:31.075370 (+    43us) tablet.cc:1370] finished BulkCheckPresence
0812 06:36:31.075385 (+    15us) tablet.cc:1372] starting ApplyRowOperation cycle
0812 06:36:31.075588 (+   203us) tablet.cc:1383] finished ApplyRowOperation cycle
0812 06:36:31.075623 (+    35us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0812 06:36:31.075628 (+     5us) write_op.cc:312] APPLY: finished
0812 06:36:31.075713 (+    85us) log.cc:844] Serialized 77 byte log entry
0812 06:36:31.075881 (+   168us) write_op.cc:489] Releasing partition, row and schema locks
0812 06:36:31.075900 (+    19us) write_op.cc:454] Released schema lock
0812 06:36:31.075908 (+     8us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":4336,"num_ops":10,"prepare.queue_time_us":2375,"prepare.run_cpu_time_us":508,"prepare.run_wall_time_us":2349,"replication_time_us":2207171,"thread_start_us":165,"threads_started":2,"wal-append.queue_time_us":159}]]}
W20260812 06:36:31.077912  6404 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:33412: Illegal state: replica 7962f587ccd842ad8a3f6ed9a5d7707e is not leader of this config: current role FOLLOWER
W20260812 06:36:31.083442  7049 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50492: Illegal state: replica 2df4308c852240859ada53d216ab69ae is not leader of this config: current role FOLLOWER
W20260812 06:36:31.086882  6621 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:42408: Illegal state: replica 42895a9bcba64aa484f6509605035d02 is not leader of this config: current role FOLLOWER
I20260812 06:36:31.093039  6046 raft_consensus.cc:1275] T 7065bb403e3047dd9a0853c49318ef1f P ec6c8d818e744cdbb6b0d15b321010e0 [term 10 FOLLOWER]: Refusing update from remote peer 222b20de5a1145c6b8a11df8374e8790: Log matching property violated. Preceding OpId in replica: term: 9 index: 980. Preceding OpId from leader: term: 10 index: 1095. (index mismatch)
I20260812 06:36:31.093014  7321 raft_consensus.cc:1275] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08 [term 10 FOLLOWER]: Refusing update from remote peer 222b20de5a1145c6b8a11df8374e8790: Log matching property violated. Preceding OpId in replica: term: 9 index: 1014. Preceding OpId from leader: term: 10 index: 1095. (index mismatch)
I20260812 06:36:31.093358  7099 raft_consensus.cc:1275] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [term 10 FOLLOWER]: Refusing update from remote peer 222b20de5a1145c6b8a11df8374e8790: Log matching property violated. Preceding OpId in replica: term: 9 index: 1100. Preceding OpId from leader: term: 10 index: 1095. (term mismatch)
I20260812 06:36:31.093338  6249 raft_consensus.cc:1275] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b [term 10 FOLLOWER]: Refusing update from remote peer 222b20de5a1145c6b8a11df8374e8790: Log matching property violated. Preceding OpId in replica: term: 9 index: 1046. Preceding OpId from leader: term: 10 index: 1095. (index mismatch)
I20260812 06:36:31.093426  7099 pending_rounds.cc:85] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae: Aborting all ops after (but not including) 1094
I20260812 06:36:31.093454  7099 pending_rounds.cc:107] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae: Aborting uncommitted WRITE_OP operation due to leader change: 9.1095
I20260812 06:36:31.093569  6703 raft_consensus.cc:1275] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [term 10 FOLLOWER]: Refusing update from remote peer 222b20de5a1145c6b8a11df8374e8790: Log matching property violated. Preceding OpId in replica: term: 9 index: 1100. Preceding OpId from leader: term: 10 index: 1095. (term mismatch)
I20260812 06:36:31.093640  6703 pending_rounds.cc:85] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02: Aborting all ops after (but not including) 1094
I20260812 06:36:31.093665  6703 pending_rounds.cc:107] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02: Aborting uncommitted WRITE_OP operation due to leader change: 9.1095
I20260812 06:36:31.093696  7099 pending_rounds.cc:107] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae: Aborting uncommitted WRITE_OP operation due to leader change: 9.1096
I20260812 06:36:31.093801  6472 raft_consensus.cc:1275] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [term 10 FOLLOWER]: Refusing update from remote peer 222b20de5a1145c6b8a11df8374e8790: Log matching property violated. Preceding OpId in replica: term: 9 index: 1100. Preceding OpId from leader: term: 10 index: 1095. (term mismatch)
I20260812 06:36:31.093856  6472 pending_rounds.cc:85] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e: Aborting all ops after (but not including) 1094
I20260812 06:36:31.093881  6472 pending_rounds.cc:107] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e: Aborting uncommitted WRITE_OP operation due to leader change: 9.1095
I20260812 06:36:31.093883  6703 pending_rounds.cc:107] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02: Aborting uncommitted WRITE_OP operation due to leader change: 9.1096
I20260812 06:36:31.093837  8161 consensus_queue.cc:1048] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" member_type: VOTER last_known_addr { host: "127.4.177.7" port: 46299 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1094, Last known committed idx: 1006, Time since last communication: 0.000s
I20260812 06:36:31.093916  7099 pending_rounds.cc:107] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae: Aborting uncommitted WRITE_OP operation due to leader change: 9.1097
W20260812 06:36:31.093999  6472 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:33412: Aborted: Op aborted by new leader
I20260812 06:36:31.094022  6703 pending_rounds.cc:107] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02: Aborting uncommitted WRITE_OP operation due to leader change: 9.1097
I20260812 06:36:31.094069  7099 pending_rounds.cc:107] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae: Aborting uncommitted WRITE_OP operation due to leader change: 9.1098
I20260812 06:36:31.094129  6472 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:33412 (ReqId={client: 3067f3aef2a44cc4bb7f2ccd6acf83db, seq_no=1090, attempt_no=0}) took 2232 ms. Trace:
I20260812 06:36:31.094175  6703 pending_rounds.cc:107] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02: Aborting uncommitted WRITE_OP operation due to leader change: 9.1098
I20260812 06:36:31.094185  7099 pending_rounds.cc:107] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae: Aborting uncommitted WRITE_OP operation due to leader change: 9.1099
I20260812 06:36:31.094333  6703 pending_rounds.cc:107] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02: Aborting uncommitted WRITE_OP operation due to leader change: 9.1099
I20260812 06:36:31.094354  7099 pending_rounds.cc:107] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae: Aborting uncommitted WRITE_OP operation due to leader change: 9.1100
I20260812 06:36:31.094195  6472 rpcz_store.cc:276] 0812 06:36:28.861262 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 06:36:28.861300 (+    38us) service_pool.cc:224] Handling call
0812 06:36:31.094119 (+2232819us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0812 06:36:28.866269 (+     0us) write_op.cc:183] PREPARE: starting on tablet 7065bb403e3047dd9a0853c49318ef1f
0812 06:36:28.866337 (+    68us) write_op.cc:432] Acquiring schema lock in shared mode
0812 06:36:28.866340 (+     3us) write_op.cc:435] Acquired schema lock
0812 06:36:28.866341 (+     1us) tablet.cc:662] Decoding operations
0812 06:36:28.866421 (+    80us) write_op.cc:620] Acquiring the partition lock for write op
0812 06:36:28.866427 (+     6us) write_op.cc:641] Partition lock acquired for write op
0812 06:36:28.866429 (+     2us) tablet.cc:685] Acquiring locks for 10 operations
0812 06:36:28.866489 (+    60us) tablet.cc:701] Row locks acquired
0812 06:36:28.866490 (+     1us) write_op.cc:260] PREPARE: finished
0812 06:36:28.866516 (+    26us) write_op.cc:270] Start()
0812 06:36:28.866535 (+    19us) write_op.cc:276] Timestamp: P: 1786516588866216 usec, L: 0
0812 06:36:28.866536 (+     1us) op_driver.cc:348] REPLICATION: starting
0812 06:36:28.866663 (+   127us) log.cc:844] Serialized 696 byte log entry
0812 06:36:31.093954 (+2227291us) op_driver.cc:464] REPLICATION: finished
0812 06:36:31.093959 (+     5us) op_driver.cc:389] HandleFailure(Aborted: Op aborted by new leader)
0812 06:36:31.093963 (+     4us) write_op.cc:489] Releasing partition, row and schema locks
0812 06:36:31.093989 (+    26us) write_op.cc:454] Released schema lock
0812 06:36:31.093995 (+     6us) write_op.cc:335] FINISH: Op aborted
Metrics: {"child_traces":[["op",{"num_ops":10,"prepare.queue_time_us":4773,"prepare.run_cpu_time_us":436,"prepare.run_wall_time_us":435,"replication_time_us":2227377,"spinlock_wait_cycles":46464}]]}
I20260812 06:36:31.094455  6703 pending_rounds.cc:107] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02: Aborting uncommitted WRITE_OP operation due to leader change: 9.1100
I20260812 06:36:31.094545  6472 pending_rounds.cc:107] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e: Aborting uncommitted WRITE_OP operation due to leader change: 9.1096
W20260812 06:36:31.094663  6472 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:33412: Aborted: Op aborted by new leader
I20260812 06:36:31.094762  6472 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:33412 (ReqId={client: 3067f3aef2a44cc4bb7f2ccd6acf83db, seq_no=1089, attempt_no=0}) took 2233 ms. Trace:
I20260812 06:36:31.094808  6472 rpcz_store.cc:276] 0812 06:36:28.861195 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 06:36:28.861448 (+   253us) service_pool.cc:224] Handling call
0812 06:36:31.094753 (+2233305us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0812 06:36:28.866727 (+     0us) write_op.cc:183] PREPARE: starting on tablet 7065bb403e3047dd9a0853c49318ef1f
0812 06:36:28.866801 (+    74us) write_op.cc:432] Acquiring schema lock in shared mode
0812 06:36:28.866804 (+     3us) write_op.cc:435] Acquired schema lock
0812 06:36:28.866806 (+     2us) tablet.cc:662] Decoding operations
0812 06:36:28.866904 (+    98us) write_op.cc:620] Acquiring the partition lock for write op
0812 06:36:28.866910 (+     6us) write_op.cc:641] Partition lock acquired for write op
0812 06:36:28.866912 (+     2us) tablet.cc:685] Acquiring locks for 10 operations
0812 06:36:28.866980 (+    68us) tablet.cc:701] Row locks acquired
0812 06:36:28.866982 (+     2us) write_op.cc:260] PREPARE: finished
0812 06:36:28.867085 (+   103us) write_op.cc:270] Start()
0812 06:36:28.867101 (+    16us) write_op.cc:276] Timestamp: P: 1786516588866731 usec, L: 0
0812 06:36:28.867102 (+     1us) op_driver.cc:348] REPLICATION: starting
0812 06:36:28.867218 (+   116us) log.cc:844] Serialized 696 byte log entry
0812 06:36:31.094625 (+2227407us) op_driver.cc:464] REPLICATION: finished
0812 06:36:31.094631 (+     6us) op_driver.cc:389] HandleFailure(Aborted: Op aborted by new leader)
0812 06:36:31.094634 (+     3us) write_op.cc:489] Releasing partition, row and schema locks
0812 06:36:31.094656 (+    22us) write_op.cc:454] Released schema lock
0812 06:36:31.094661 (+     5us) write_op.cc:335] FINISH: Op aborted
Metrics: {"child_traces":[["op",{"num_ops":10,"prepare.queue_time_us":5032,"prepare.run_cpu_time_us":534,"prepare.run_wall_time_us":534,"replication_time_us":2227495,"spinlock_wait_cycles":33024}]]}
I20260812 06:36:31.095074  6472 pending_rounds.cc:107] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e: Aborting uncommitted WRITE_OP operation due to leader change: 9.1097
I20260812 06:36:31.095134  8144 consensus_queue.cc:1048] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8d90d61da62549208fe117b4384a469b" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 41539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1094, Last known committed idx: 1046, Time since last communication: 0.000s
W20260812 06:36:31.095199  6472 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:33412: Aborted: Op aborted by new leader
I20260812 06:36:31.095295  6472 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:33412 (ReqId={client: 3067f3aef2a44cc4bb7f2ccd6acf83db, seq_no=1091, attempt_no=0}) took 2233 ms. Trace:
I20260812 06:36:31.095338  6472 rpcz_store.cc:276] 0812 06:36:28.861991 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 06:36:28.862047 (+    56us) service_pool.cc:224] Handling call
0812 06:36:31.095287 (+2233240us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0812 06:36:28.867288 (+     0us) write_op.cc:183] PREPARE: starting on tablet 7065bb403e3047dd9a0853c49318ef1f
0812 06:36:28.868099 (+   811us) write_op.cc:432] Acquiring schema lock in shared mode
0812 06:36:28.868105 (+     6us) write_op.cc:435] Acquired schema lock
0812 06:36:28.868108 (+     3us) tablet.cc:662] Decoding operations
0812 06:36:28.868205 (+    97us) write_op.cc:620] Acquiring the partition lock for write op
0812 06:36:28.868211 (+     6us) write_op.cc:641] Partition lock acquired for write op
0812 06:36:28.868214 (+     3us) tablet.cc:685] Acquiring locks for 10 operations
0812 06:36:28.868273 (+    59us) tablet.cc:701] Row locks acquired
0812 06:36:28.868274 (+     1us) write_op.cc:260] PREPARE: finished
0812 06:36:28.868301 (+    27us) write_op.cc:270] Start()
0812 06:36:28.868319 (+    18us) write_op.cc:276] Timestamp: P: 1786516588868001 usec, L: 0
0812 06:36:28.868321 (+     2us) op_driver.cc:348] REPLICATION: starting
0812 06:36:28.868424 (+   103us) log.cc:844] Serialized 696 byte log entry
0812 06:36:31.095162 (+2226738us) op_driver.cc:464] REPLICATION: finished
0812 06:36:31.095168 (+     6us) op_driver.cc:389] HandleFailure(Aborted: Op aborted by new leader)
0812 06:36:31.095171 (+     3us) write_op.cc:489] Releasing partition, row and schema locks
0812 06:36:31.095191 (+    20us) write_op.cc:454] Released schema lock
0812 06:36:31.095196 (+     5us) write_op.cc:335] FINISH: Op aborted
Metrics: {"child_traces":[["op",{"num_ops":10,"prepare.queue_time_us":4961,"prepare.run_cpu_time_us":459,"prepare.run_wall_time_us":1179,"replication_time_us":2226808}]]}
I20260812 06:36:31.095458  8141 consensus_queue.cc:1048] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [LEADER]: Connected to new peer: Peer: permanent_uuid: "42895a9bcba64aa484f6509605035d02" member_type: VOTER last_known_addr { host: "127.4.177.4" port: 38181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1094, Last known committed idx: 1092, Time since last communication: 0.000s
I20260812 06:36:31.095569  6472 pending_rounds.cc:107] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e: Aborting uncommitted WRITE_OP operation due to leader change: 9.1098
I20260812 06:36:31.095580  8141 consensus_queue.cc:1243] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [LEADER]: Peer 42895a9bcba64aa484f6509605035d02 log is divergent from this leader: its last log entry 9.1094 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1092
W20260812 06:36:31.095672  6472 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:33412: Aborted: Op aborted by new leader
I20260812 06:36:31.095762  6472 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:33412 (ReqId={client: 3067f3aef2a44cc4bb7f2ccd6acf83db, seq_no=1092, attempt_no=0}) took 2231 ms. Trace:
I20260812 06:36:31.095803  8141 consensus_queue.cc:1048] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2df4308c852240859ada53d216ab69ae" member_type: VOTER last_known_addr { host: "127.4.177.6" port: 34269 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1094, Last known committed idx: 1092, Time since last communication: 0.000s
I20260812 06:36:31.095883  8141 consensus_queue.cc:1243] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [LEADER]: Peer 2df4308c852240859ada53d216ab69ae log is divergent from this leader: its last log entry 9.1094 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1092
I20260812 06:36:31.095798  6472 rpcz_store.cc:276] 0812 06:36:28.864715 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 06:36:28.864786 (+    71us) service_pool.cc:224] Handling call
0812 06:36:31.095755 (+2230969us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0812 06:36:28.868494 (+     0us) write_op.cc:183] PREPARE: starting on tablet 7065bb403e3047dd9a0853c49318ef1f
0812 06:36:28.868558 (+    64us) write_op.cc:432] Acquiring schema lock in shared mode
0812 06:36:28.868560 (+     2us) write_op.cc:435] Acquired schema lock
0812 06:36:28.868562 (+     2us) tablet.cc:662] Decoding operations
0812 06:36:28.868638 (+    76us) write_op.cc:620] Acquiring the partition lock for write op
0812 06:36:28.868643 (+     5us) write_op.cc:641] Partition lock acquired for write op
0812 06:36:28.868646 (+     3us) tablet.cc:685] Acquiring locks for 10 operations
0812 06:36:28.868710 (+    64us) tablet.cc:701] Row locks acquired
0812 06:36:28.868711 (+     1us) write_op.cc:260] PREPARE: finished
0812 06:36:28.868734 (+    23us) write_op.cc:270] Start()
0812 06:36:28.868755 (+    21us) write_op.cc:276] Timestamp: P: 1786516588868435 usec, L: 0
0812 06:36:28.868758 (+     3us) op_driver.cc:348] REPLICATION: starting
0812 06:36:28.871844 (+  3086us) log.cc:844] Serialized 686 byte log entry
0812 06:36:31.095636 (+2223792us) op_driver.cc:464] REPLICATION: finished
0812 06:36:31.095641 (+     5us) op_driver.cc:389] HandleFailure(Aborted: Op aborted by new leader)
0812 06:36:31.095644 (+     3us) write_op.cc:489] Releasing partition, row and schema locks
0812 06:36:31.095664 (+    20us) write_op.cc:454] Released schema lock
0812 06:36:31.095668 (+     4us) write_op.cc:335] FINISH: Op aborted
Metrics: {"child_traces":[["op",{"num_ops":10,"prepare.queue_time_us":3471,"prepare.run_cpu_time_us":465,"prepare.run_wall_time_us":3409,"replication_time_us":2226849}]]}
I20260812 06:36:31.095937  8144 consensus_queue.cc:1048] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 41057 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1094, Last known committed idx: 973, Time since last communication: 0.000s
I20260812 06:36:31.096015  6472 pending_rounds.cc:107] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e: Aborting uncommitted WRITE_OP operation due to leader change: 9.1099
W20260812 06:36:31.096117  6472 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:33412: Aborted: Op aborted by new leader
I20260812 06:36:31.096201  6472 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:33412 (ReqId={client: 3067f3aef2a44cc4bb7f2ccd6acf83db, seq_no=1093, attempt_no=0}) took 2228 ms. Trace:
I20260812 06:36:31.096241  6472 rpcz_store.cc:276] 0812 06:36:28.868391 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 06:36:28.868431 (+    40us) service_pool.cc:224] Handling call
0812 06:36:31.096194 (+2227763us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0812 06:36:28.871933 (+     0us) write_op.cc:183] PREPARE: starting on tablet 7065bb403e3047dd9a0853c49318ef1f
0812 06:36:28.872014 (+    81us) write_op.cc:432] Acquiring schema lock in shared mode
0812 06:36:28.872017 (+     3us) write_op.cc:435] Acquired schema lock
0812 06:36:28.872018 (+     1us) tablet.cc:662] Decoding operations
0812 06:36:28.872099 (+    81us) write_op.cc:620] Acquiring the partition lock for write op
0812 06:36:28.872105 (+     6us) write_op.cc:641] Partition lock acquired for write op
0812 06:36:28.872108 (+     3us) tablet.cc:685] Acquiring locks for 10 operations
0812 06:36:28.872166 (+    58us) tablet.cc:701] Row locks acquired
0812 06:36:28.872168 (+     2us) write_op.cc:260] PREPARE: finished
0812 06:36:28.872198 (+    30us) write_op.cc:270] Start()
0812 06:36:28.872214 (+    16us) write_op.cc:276] Timestamp: P: 1786516588871895 usec, L: 0
0812 06:36:28.872215 (+     1us) op_driver.cc:348] REPLICATION: starting
0812 06:36:28.872318 (+   103us) log.cc:844] Serialized 696 byte log entry
0812 06:36:31.096081 (+2223763us) op_driver.cc:464] REPLICATION: finished
0812 06:36:31.096086 (+     5us) op_driver.cc:389] HandleFailure(Aborted: Op aborted by new leader)
0812 06:36:31.096089 (+     3us) write_op.cc:489] Releasing partition, row and schema locks
0812 06:36:31.096108 (+    19us) write_op.cc:454] Released schema lock
0812 06:36:31.096113 (+     5us) write_op.cc:335] FINISH: Op aborted
Metrics: {"child_traces":[["op",{"num_ops":10,"prepare.queue_time_us":3274,"prepare.run_cpu_time_us":439,"prepare.run_wall_time_us":439,"replication_time_us":2223836}]]}
I20260812 06:36:31.096457  6472 pending_rounds.cc:107] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e: Aborting uncommitted WRITE_OP operation due to leader change: 9.1100
W20260812 06:36:31.096562  6472 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:33412: Aborted: Op aborted by new leader
I20260812 06:36:31.096648  6472 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:33412 (ReqId={client: 3067f3aef2a44cc4bb7f2ccd6acf83db, seq_no=1094, attempt_no=0}) took 2226 ms. Trace:
I20260812 06:36:31.096680  6472 rpcz_store.cc:276] 0812 06:36:28.869769 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 06:36:28.870235 (+   466us) service_pool.cc:224] Handling call
0812 06:36:31.096642 (+2226407us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0812 06:36:28.872405 (+     0us) write_op.cc:183] PREPARE: starting on tablet 7065bb403e3047dd9a0853c49318ef1f
0812 06:36:28.872480 (+    75us) write_op.cc:432] Acquiring schema lock in shared mode
0812 06:36:28.872483 (+     3us) write_op.cc:435] Acquired schema lock
0812 06:36:28.872485 (+     2us) tablet.cc:662] Decoding operations
0812 06:36:28.872579 (+    94us) write_op.cc:620] Acquiring the partition lock for write op
0812 06:36:28.872585 (+     6us) write_op.cc:641] Partition lock acquired for write op
0812 06:36:28.872588 (+     3us) tablet.cc:685] Acquiring locks for 10 operations
0812 06:36:28.872649 (+    61us) tablet.cc:701] Row locks acquired
0812 06:36:28.872650 (+     1us) write_op.cc:260] PREPARE: finished
0812 06:36:28.872678 (+    28us) write_op.cc:270] Start()
0812 06:36:28.872700 (+    22us) write_op.cc:276] Timestamp: P: 1786516588872376 usec, L: 0
0812 06:36:28.872703 (+     3us) op_driver.cc:348] REPLICATION: starting
0812 06:36:28.872806 (+   103us) log.cc:844] Serialized 696 byte log entry
0812 06:36:31.096519 (+2223713us) op_driver.cc:464] REPLICATION: finished
0812 06:36:31.096524 (+     5us) op_driver.cc:389] HandleFailure(Aborted: Op aborted by new leader)
0812 06:36:31.096527 (+     3us) write_op.cc:489] Releasing partition, row and schema locks
0812 06:36:31.096544 (+    17us) write_op.cc:454] Released schema lock
0812 06:36:31.096560 (+    16us) write_op.cc:335] FINISH: Op aborted
Metrics: {"child_traces":[["op",{"num_ops":10,"prepare.queue_time_us":1877,"prepare.run_cpu_time_us":480,"prepare.run_wall_time_us":480,"replication_time_us":2223788,"spinlock_wait_cycles":28544}]]}
I20260812 06:36:31.098574  8141 consensus_queue.cc:1048] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 40897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1094, Last known committed idx: 1093, Time since last communication: 0.000s
I20260812 06:36:31.098685  8141 consensus_queue.cc:1243] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [LEADER]: Peer 7962f587ccd842ad8a3f6ed9a5d7707e log is divergent from this leader: its last log entry 9.1094 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1093
I20260812 06:36:31.099448  6703 raft_consensus.cc:1217] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [term 10 FOLLOWER]: Deduplicated request from leader. Original: 9.1092->[9.1093-10.1095]   Dedup: 9.1093->[10.1094-10.1095]
I20260812 06:36:31.099534  6703 pending_rounds.cc:85] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02: Aborting all ops after (but not including) 1093
I20260812 06:36:31.099570  6703 pending_rounds.cc:107] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02: Aborting uncommitted WRITE_OP operation due to leader change: 9.1094
I20260812 06:36:31.101483  6472 pending_rounds.cc:85] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e: Aborting all ops after (but not including) 1093
I20260812 06:36:31.101536  6472 pending_rounds.cc:107] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e: Aborting uncommitted WRITE_OP operation due to leader change: 9.1094
W20260812 06:36:31.101653  6472 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:33412: Aborted: Op aborted by new leader
I20260812 06:36:31.101764  6472 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:33412 (ReqId={client: 3067f3aef2a44cc4bb7f2ccd6acf83db, seq_no=1088, attempt_no=0}) took 2240 ms. Trace:
I20260812 06:36:31.101805  6472 rpcz_store.cc:276] 0812 06:36:28.861102 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 06:36:28.861167 (+    65us) service_pool.cc:224] Handling call
0812 06:36:31.101754 (+2240587us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0812 06:36:28.865768 (+     0us) write_op.cc:183] PREPARE: starting on tablet 7065bb403e3047dd9a0853c49318ef1f
0812 06:36:28.865842 (+    74us) write_op.cc:432] Acquiring schema lock in shared mode
0812 06:36:28.865845 (+     3us) write_op.cc:435] Acquired schema lock
0812 06:36:28.865846 (+     1us) tablet.cc:662] Decoding operations
0812 06:36:28.865942 (+    96us) write_op.cc:620] Acquiring the partition lock for write op
0812 06:36:28.865947 (+     5us) write_op.cc:641] Partition lock acquired for write op
0812 06:36:28.865950 (+     3us) tablet.cc:685] Acquiring locks for 10 operations
0812 06:36:28.866016 (+    66us) tablet.cc:701] Row locks acquired
0812 06:36:28.866017 (+     1us) write_op.cc:260] PREPARE: finished
0812 06:36:28.866043 (+    26us) write_op.cc:270] Start()
0812 06:36:28.866060 (+    17us) write_op.cc:276] Timestamp: P: 1786516588865743 usec, L: 0
0812 06:36:28.866062 (+     2us) op_driver.cc:348] REPLICATION: starting
0812 06:36:28.866176 (+   114us) log.cc:844] Serialized 686 byte log entry
0812 06:36:31.101613 (+2235437us) op_driver.cc:464] REPLICATION: finished
0812 06:36:31.101620 (+     7us) op_driver.cc:389] HandleFailure(Aborted: Op aborted by new leader)
0812 06:36:31.101623 (+     3us) write_op.cc:489] Releasing partition, row and schema locks
0812 06:36:31.101644 (+    21us) write_op.cc:454] Released schema lock
0812 06:36:31.101649 (+     5us) write_op.cc:335] FINISH: Op aborted
Metrics: {"child_traces":[["op",{"num_ops":10,"prepare.queue_time_us":4341,"prepare.run_cpu_time_us":452,"prepare.run_wall_time_us":452,"replication_time_us":2235517,"spinlock_wait_cycles":3840}]]}
I20260812 06:36:31.105655  7099 raft_consensus.cc:1217] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [term 10 FOLLOWER]: Deduplicated request from leader. Original: 9.1092->[9.1093-10.1095]   Dedup: 9.1093->[10.1094-10.1095]
I20260812 06:36:31.105803  7099 pending_rounds.cc:85] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae: Aborting all ops after (but not including) 1093
I20260812 06:36:31.105839  7099 pending_rounds.cc:107] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae: Aborting uncommitted WRITE_OP operation due to leader change: 9.1094
I20260812 06:36:32.150027  4804 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskCnZu03/build/debug/bin/kudu with pid 6795
W20260812 06:36:32.163784  7401 meta_cache.cc:302] tablet 7065bb403e3047dd9a0853c49318ef1f: replica 222b20de5a1145c6b8a11df8374e8790 (127.4.177.5:43521) has failed: Network error: recv got EOF from 127.4.177.5:43521 (error 108)
W20260812 06:36:32.166538  5939 connection.cc:466] server connection from 127.4.177.5:43279 torn down before Call kudu.consensus.ConsensusService.UpdateConsensus from 127.4.177.5:43279 (request call id 37) could send its response
W20260812 06:36:32.171765  6150 connection.cc:466] server connection from 127.4.177.5:52783 torn down before Call kudu.consensus.ConsensusService.UpdateConsensus from 127.4.177.5:52783 (request call id 98) could send its response
W20260812 06:36:32.173151  7210 connection.cc:466] server connection from 127.4.177.5:44367 torn down before Call kudu.consensus.ConsensusService.UpdateConsensus from 127.4.177.5:44367 (request call id 174) could send its response
W20260812 06:36:32.177040  6365 connection.cc:466] server connection from 127.4.177.5:43731 torn down before Call kudu.consensus.ConsensusService.UpdateConsensus from 127.4.177.5:43731 (request call id 164) could send its response
I20260812 06:36:33.207158  8264 raft_consensus.cc:493] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 222b20de5a1145c6b8a11df8374e8790)
I20260812 06:36:33.207311  8264 raft_consensus.cc:515] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df4308c852240859ada53d216ab69ae" member_type: VOTER last_known_addr { host: "127.4.177.6" port: 34269 } } peers { permanent_uuid: "42895a9bcba64aa484f6509605035d02" member_type: VOTER last_known_addr { host: "127.4.177.4" port: 38181 } } peers { permanent_uuid: "222b20de5a1145c6b8a11df8374e8790" member_type: VOTER last_known_addr { host: "127.4.177.5" port: 43521 } } peers { permanent_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 40897 } } peers { permanent_uuid: "8d90d61da62549208fe117b4384a469b" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 41539 } } peers { permanent_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" member_type: VOTER last_known_addr { host: "127.4.177.7" port: 46299 } } peers { permanent_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 41057 } }
I20260812 06:36:33.207943  8264 leader_election.cc:290] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 2df4308c852240859ada53d216ab69ae (127.4.177.6:34269), 42895a9bcba64aa484f6509605035d02 (127.4.177.4:38181), 222b20de5a1145c6b8a11df8374e8790 (127.4.177.5:43521), 7962f587ccd842ad8a3f6ed9a5d7707e (127.4.177.3:40897), b037d17b21f8422fa5e41b70eb4dcd08 (127.4.177.7:46299), ec6c8d818e744cdbb6b0d15b321010e0 (127.4.177.1:41057)
I20260812 06:36:33.208400  7099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "8d90d61da62549208fe117b4384a469b" candidate_term: 11 candidate_status { last_received { term: 10 index: 1254 } } ignore_live_leader: false dest_uuid: "2df4308c852240859ada53d216ab69ae" is_pre_election: true
I20260812 06:36:33.208420  7320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "8d90d61da62549208fe117b4384a469b" candidate_term: 11 candidate_status { last_received { term: 10 index: 1254 } } ignore_live_leader: false dest_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" is_pre_election: true
I20260812 06:36:33.208472  6703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "8d90d61da62549208fe117b4384a469b" candidate_term: 11 candidate_status { last_received { term: 10 index: 1254 } } ignore_live_leader: false dest_uuid: "42895a9bcba64aa484f6509605035d02" is_pre_election: true
I20260812 06:36:33.209023  6046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "8d90d61da62549208fe117b4384a469b" candidate_term: 11 candidate_status { last_received { term: 10 index: 1254 } } ignore_live_leader: false dest_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" is_pre_election: true
I20260812 06:36:33.209157  6472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "8d90d61da62549208fe117b4384a469b" candidate_term: 11 candidate_status { last_received { term: 10 index: 1254 } } ignore_live_leader: false dest_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" is_pre_election: true
W20260812 06:36:33.209237  6150 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.4.177.5:43521: connect: Connection refused (error 111)
I20260812 06:36:33.209555  6150 leader_election.cc:304] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 5 responses out of 7 voters: 1 yes votes; 4 no votes. yes voters: 8d90d61da62549208fe117b4384a469b; no voters: 2df4308c852240859ada53d216ab69ae, 42895a9bcba64aa484f6509605035d02, b037d17b21f8422fa5e41b70eb4dcd08, ec6c8d818e744cdbb6b0d15b321010e0
I20260812 06:36:33.209794  8264 raft_consensus.cc:2749] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260812 06:36:33.210121  6150 leader_election.cc:336] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 222b20de5a1145c6b8a11df8374e8790 (127.4.177.5:43521): Network error: Client connection negotiation failed: client connection to 127.4.177.5:43521: connect: Connection refused (error 111)
I20260812 06:36:33.327482  8269 raft_consensus.cc:493] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 222b20de5a1145c6b8a11df8374e8790)
I20260812 06:36:33.327620  8270 raft_consensus.cc:493] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 222b20de5a1145c6b8a11df8374e8790)
I20260812 06:36:33.327637  8269 raft_consensus.cc:515] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df4308c852240859ada53d216ab69ae" member_type: VOTER last_known_addr { host: "127.4.177.6" port: 34269 } } peers { permanent_uuid: "42895a9bcba64aa484f6509605035d02" member_type: VOTER last_known_addr { host: "127.4.177.4" port: 38181 } } peers { permanent_uuid: "222b20de5a1145c6b8a11df8374e8790" member_type: VOTER last_known_addr { host: "127.4.177.5" port: 43521 } } peers { permanent_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 40897 } } peers { permanent_uuid: "8d90d61da62549208fe117b4384a469b" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 41539 } } peers { permanent_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" member_type: VOTER last_known_addr { host: "127.4.177.7" port: 46299 } } peers { permanent_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 41057 } }
I20260812 06:36:33.327785  8270 raft_consensus.cc:515] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df4308c852240859ada53d216ab69ae" member_type: VOTER last_known_addr { host: "127.4.177.6" port: 34269 } } peers { permanent_uuid: "42895a9bcba64aa484f6509605035d02" member_type: VOTER last_known_addr { host: "127.4.177.4" port: 38181 } } peers { permanent_uuid: "222b20de5a1145c6b8a11df8374e8790" member_type: VOTER last_known_addr { host: "127.4.177.5" port: 43521 } } peers { permanent_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 40897 } } peers { permanent_uuid: "8d90d61da62549208fe117b4384a469b" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 41539 } } peers { permanent_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" member_type: VOTER last_known_addr { host: "127.4.177.7" port: 46299 } } peers { permanent_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 41057 } }
I20260812 06:36:33.328738  8270 leader_election.cc:290] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 42895a9bcba64aa484f6509605035d02 (127.4.177.4:38181), 222b20de5a1145c6b8a11df8374e8790 (127.4.177.5:43521), 7962f587ccd842ad8a3f6ed9a5d7707e (127.4.177.3:40897), 8d90d61da62549208fe117b4384a469b (127.4.177.2:41539), b037d17b21f8422fa5e41b70eb4dcd08 (127.4.177.7:46299), ec6c8d818e744cdbb6b0d15b321010e0 (127.4.177.1:41057)
I20260812 06:36:33.328847  8269 leader_election.cc:290] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 2df4308c852240859ada53d216ab69ae (127.4.177.6:34269), 222b20de5a1145c6b8a11df8374e8790 (127.4.177.5:43521), 7962f587ccd842ad8a3f6ed9a5d7707e (127.4.177.3:40897), 8d90d61da62549208fe117b4384a469b (127.4.177.2:41539), b037d17b21f8422fa5e41b70eb4dcd08 (127.4.177.7:46299), ec6c8d818e744cdbb6b0d15b321010e0 (127.4.177.1:41057)
I20260812 06:36:33.329414  6703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "2df4308c852240859ada53d216ab69ae" candidate_term: 11 candidate_status { last_received { term: 10 index: 1307 } } ignore_live_leader: false dest_uuid: "42895a9bcba64aa484f6509605035d02" is_pre_election: true
I20260812 06:36:33.329566  6703 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2df4308c852240859ada53d216ab69ae in term 10.
I20260812 06:36:33.329550  6479 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "2df4308c852240859ada53d216ab69ae" candidate_term: 11 candidate_status { last_received { term: 10 index: 1307 } } ignore_live_leader: false dest_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" is_pre_election: true
I20260812 06:36:33.329550  6472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "42895a9bcba64aa484f6509605035d02" candidate_term: 11 candidate_status { last_received { term: 10 index: 1307 } } ignore_live_leader: false dest_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" is_pre_election: true
I20260812 06:36:33.329666  6047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "42895a9bcba64aa484f6509605035d02" candidate_term: 11 candidate_status { last_received { term: 10 index: 1307 } } ignore_live_leader: false dest_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" is_pre_election: true
I20260812 06:36:33.329905  6046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "2df4308c852240859ada53d216ab69ae" candidate_term: 11 candidate_status { last_received { term: 10 index: 1307 } } ignore_live_leader: false dest_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" is_pre_election: true
W20260812 06:36:33.330142  7020 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.4.177.5:43521: connect: Connection refused (error 111) [suppressed 1 similar messages]
I20260812 06:36:33.330204  7320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "2df4308c852240859ada53d216ab69ae" candidate_term: 11 candidate_status { last_received { term: 10 index: 1307 } } ignore_live_leader: false dest_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" is_pre_election: true
I20260812 06:36:33.330436  6247 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "2df4308c852240859ada53d216ab69ae" candidate_term: 11 candidate_status { last_received { term: 10 index: 1307 } } ignore_live_leader: false dest_uuid: "8d90d61da62549208fe117b4384a469b" is_pre_election: true
I20260812 06:36:33.330575  6247 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2df4308c852240859ada53d216ab69ae in term 10.
I20260812 06:36:33.330669  7099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "42895a9bcba64aa484f6509605035d02" candidate_term: 11 candidate_status { last_received { term: 10 index: 1307 } } ignore_live_leader: false dest_uuid: "2df4308c852240859ada53d216ab69ae" is_pre_election: true
I20260812 06:36:33.330585  6249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "42895a9bcba64aa484f6509605035d02" candidate_term: 11 candidate_status { last_received { term: 10 index: 1307 } } ignore_live_leader: false dest_uuid: "8d90d61da62549208fe117b4384a469b" is_pre_election: true
I20260812 06:36:33.330786  7099 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 42895a9bcba64aa484f6509605035d02 in term 10.
I20260812 06:36:33.330896  6249 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 42895a9bcba64aa484f6509605035d02 in term 10.
I20260812 06:36:33.330948  7320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "42895a9bcba64aa484f6509605035d02" candidate_term: 11 candidate_status { last_received { term: 10 index: 1307 } } ignore_live_leader: false dest_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" is_pre_election: true
W20260812 06:36:33.331156  6597 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.4.177.5:43521: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260812 06:36:33.331749  7020 leader_election.cc:336] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 222b20de5a1145c6b8a11df8374e8790 (127.4.177.5:43521): Network error: Client connection negotiation failed: client connection to 127.4.177.5:43521: connect: Connection refused (error 111)
I20260812 06:36:33.331840  7020 leader_election.cc:304] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 7 responses out of 7 voters: 3 yes votes; 4 no votes. yes voters: 2df4308c852240859ada53d216ab69ae, 42895a9bcba64aa484f6509605035d02, 8d90d61da62549208fe117b4384a469b; no voters: 222b20de5a1145c6b8a11df8374e8790, 7962f587ccd842ad8a3f6ed9a5d7707e, b037d17b21f8422fa5e41b70eb4dcd08, ec6c8d818e744cdbb6b0d15b321010e0
I20260812 06:36:33.332036  8270 raft_consensus.cc:2749] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260812 06:36:33.332086  6597 leader_election.cc:336] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 222b20de5a1145c6b8a11df8374e8790 (127.4.177.5:43521): Network error: Client connection negotiation failed: client connection to 127.4.177.5:43521: connect: Connection refused (error 111)
I20260812 06:36:33.332165  6597 leader_election.cc:304] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 7 responses out of 7 voters: 3 yes votes; 4 no votes. yes voters: 2df4308c852240859ada53d216ab69ae, 42895a9bcba64aa484f6509605035d02, 8d90d61da62549208fe117b4384a469b; no voters: 222b20de5a1145c6b8a11df8374e8790, 7962f587ccd842ad8a3f6ed9a5d7707e, b037d17b21f8422fa5e41b70eb4dcd08, ec6c8d818e744cdbb6b0d15b321010e0
I20260812 06:36:33.332315  8269 raft_consensus.cc:2749] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260812 06:36:33.666146  8275 raft_consensus.cc:493] T 7065bb403e3047dd9a0853c49318ef1f P ec6c8d818e744cdbb6b0d15b321010e0 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 222b20de5a1145c6b8a11df8374e8790)
I20260812 06:36:33.666404  8275 raft_consensus.cc:515] T 7065bb403e3047dd9a0853c49318ef1f P ec6c8d818e744cdbb6b0d15b321010e0 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df4308c852240859ada53d216ab69ae" member_type: VOTER last_known_addr { host: "127.4.177.6" port: 34269 } } peers { permanent_uuid: "42895a9bcba64aa484f6509605035d02" member_type: VOTER last_known_addr { host: "127.4.177.4" port: 38181 } } peers { permanent_uuid: "222b20de5a1145c6b8a11df8374e8790" member_type: VOTER last_known_addr { host: "127.4.177.5" port: 43521 } } peers { permanent_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 40897 } } peers { permanent_uuid: "8d90d61da62549208fe117b4384a469b" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 41539 } } peers { permanent_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" member_type: VOTER last_known_addr { host: "127.4.177.7" port: 46299 } } peers { permanent_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 41057 } }
I20260812 06:36:33.667428  8275 leader_election.cc:290] T 7065bb403e3047dd9a0853c49318ef1f P ec6c8d818e744cdbb6b0d15b321010e0 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 2df4308c852240859ada53d216ab69ae (127.4.177.6:34269), 42895a9bcba64aa484f6509605035d02 (127.4.177.4:38181), 222b20de5a1145c6b8a11df8374e8790 (127.4.177.5:43521), 7962f587ccd842ad8a3f6ed9a5d7707e (127.4.177.3:40897), 8d90d61da62549208fe117b4384a469b (127.4.177.2:41539), b037d17b21f8422fa5e41b70eb4dcd08 (127.4.177.7:46299)
I20260812 06:36:33.667755  7099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" candidate_term: 11 candidate_status { last_received { term: 10 index: 1181 } } ignore_live_leader: false dest_uuid: "2df4308c852240859ada53d216ab69ae" is_pre_election: true
I20260812 06:36:33.667745  6703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" candidate_term: 11 candidate_status { last_received { term: 10 index: 1181 } } ignore_live_leader: false dest_uuid: "42895a9bcba64aa484f6509605035d02" is_pre_election: true
I20260812 06:36:33.667945  7099 raft_consensus.cc:2410] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ec6c8d818e744cdbb6b0d15b321010e0 for term 11 because replica has last-logged OpId of term: 10 index: 1307, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 1181.
I20260812 06:36:33.667963  6703 raft_consensus.cc:2410] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ec6c8d818e744cdbb6b0d15b321010e0 for term 11 because replica has last-logged OpId of term: 10 index: 1307, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 1181.
I20260812 06:36:33.668253  7320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" candidate_term: 11 candidate_status { last_received { term: 10 index: 1181 } } ignore_live_leader: false dest_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" is_pre_election: true
I20260812 06:36:33.668291  6472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" candidate_term: 11 candidate_status { last_received { term: 10 index: 1181 } } ignore_live_leader: false dest_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" is_pre_election: true
I20260812 06:36:33.668427  7320 raft_consensus.cc:2410] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ec6c8d818e744cdbb6b0d15b321010e0 for term 11 because replica has last-logged OpId of term: 10 index: 1216, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 1181.
W20260812 06:36:33.668661  5945 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.4.177.5:43521: connect: Connection refused (error 111)
I20260812 06:36:33.668840  6249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" candidate_term: 11 candidate_status { last_received { term: 10 index: 1181 } } ignore_live_leader: false dest_uuid: "8d90d61da62549208fe117b4384a469b" is_pre_election: true
I20260812 06:36:33.668874  8277 raft_consensus.cc:493] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 222b20de5a1145c6b8a11df8374e8790)
I20260812 06:36:33.669004  6249 raft_consensus.cc:2410] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ec6c8d818e744cdbb6b0d15b321010e0 for term 11 because replica has last-logged OpId of term: 10 index: 1254, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 1181.
I20260812 06:36:33.668972  8277 raft_consensus.cc:515] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df4308c852240859ada53d216ab69ae" member_type: VOTER last_known_addr { host: "127.4.177.6" port: 34269 } } peers { permanent_uuid: "42895a9bcba64aa484f6509605035d02" member_type: VOTER last_known_addr { host: "127.4.177.4" port: 38181 } } peers { permanent_uuid: "222b20de5a1145c6b8a11df8374e8790" member_type: VOTER last_known_addr { host: "127.4.177.5" port: 43521 } } peers { permanent_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 40897 } } peers { permanent_uuid: "8d90d61da62549208fe117b4384a469b" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 41539 } } peers { permanent_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" member_type: VOTER last_known_addr { host: "127.4.177.7" port: 46299 } } peers { permanent_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 41057 } }
I20260812 06:36:33.669037  5945 leader_election.cc:304] T 7065bb403e3047dd9a0853c49318ef1f P ec6c8d818e744cdbb6b0d15b321010e0 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 5 responses out of 7 voters: 1 yes votes; 4 no votes. yes voters: ec6c8d818e744cdbb6b0d15b321010e0; no voters: 2df4308c852240859ada53d216ab69ae, 42895a9bcba64aa484f6509605035d02, 7962f587ccd842ad8a3f6ed9a5d7707e, b037d17b21f8422fa5e41b70eb4dcd08
I20260812 06:36:33.669200  8275 raft_consensus.cc:2749] T 7065bb403e3047dd9a0853c49318ef1f P ec6c8d818e744cdbb6b0d15b321010e0 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260812 06:36:33.669605  5945 leader_election.cc:336] T 7065bb403e3047dd9a0853c49318ef1f P ec6c8d818e744cdbb6b0d15b321010e0 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 222b20de5a1145c6b8a11df8374e8790 (127.4.177.5:43521): Network error: Client connection negotiation failed: client connection to 127.4.177.5:43521: connect: Connection refused (error 111)
I20260812 06:36:33.669948  8277 leader_election.cc:290] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 2df4308c852240859ada53d216ab69ae (127.4.177.6:34269), 42895a9bcba64aa484f6509605035d02 (127.4.177.4:38181), 222b20de5a1145c6b8a11df8374e8790 (127.4.177.5:43521), 7962f587ccd842ad8a3f6ed9a5d7707e (127.4.177.3:40897), 8d90d61da62549208fe117b4384a469b (127.4.177.2:41539), ec6c8d818e744cdbb6b0d15b321010e0 (127.4.177.1:41057)
I20260812 06:36:33.670143  7099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" candidate_term: 11 candidate_status { last_received { term: 10 index: 1216 } } ignore_live_leader: false dest_uuid: "2df4308c852240859ada53d216ab69ae" is_pre_election: true
I20260812 06:36:33.670312  6703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" candidate_term: 11 candidate_status { last_received { term: 10 index: 1216 } } ignore_live_leader: false dest_uuid: "42895a9bcba64aa484f6509605035d02" is_pre_election: true
I20260812 06:36:33.670353  7099 raft_consensus.cc:2410] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b037d17b21f8422fa5e41b70eb4dcd08 for term 11 because replica has last-logged OpId of term: 10 index: 1307, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 1216.
I20260812 06:36:33.670473  6703 raft_consensus.cc:2410] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b037d17b21f8422fa5e41b70eb4dcd08 for term 11 because replica has last-logged OpId of term: 10 index: 1307, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 1216.
W20260812 06:36:33.670876  7211 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.4.177.5:43521: connect: Connection refused (error 111)
I20260812 06:36:33.670964  6249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" candidate_term: 11 candidate_status { last_received { term: 10 index: 1216 } } ignore_live_leader: false dest_uuid: "8d90d61da62549208fe117b4384a469b" is_pre_election: true
I20260812 06:36:33.670979  6472 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" candidate_term: 11 candidate_status { last_received { term: 10 index: 1216 } } ignore_live_leader: false dest_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" is_pre_election: true
I20260812 06:36:33.671124  6249 raft_consensus.cc:2410] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b037d17b21f8422fa5e41b70eb4dcd08 for term 11 because replica has last-logged OpId of term: 10 index: 1254, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 1216.
I20260812 06:36:33.671238  6046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" candidate_term: 11 candidate_status { last_received { term: 10 index: 1216 } } ignore_live_leader: false dest_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" is_pre_election: true
I20260812 06:36:33.671373  6046 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P ec6c8d818e744cdbb6b0d15b321010e0 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b037d17b21f8422fa5e41b70eb4dcd08 in term 10.
I20260812 06:36:33.671538  7211 leader_election.cc:304] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 5 responses out of 7 voters: 1 yes votes; 4 no votes. yes voters: b037d17b21f8422fa5e41b70eb4dcd08; no voters: 2df4308c852240859ada53d216ab69ae, 42895a9bcba64aa484f6509605035d02, 7962f587ccd842ad8a3f6ed9a5d7707e, 8d90d61da62549208fe117b4384a469b
I20260812 06:36:33.671717  8277 raft_consensus.cc:2749] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260812 06:36:33.672096  7211 leader_election.cc:336] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 222b20de5a1145c6b8a11df8374e8790 (127.4.177.5:43521): Network error: Client connection negotiation failed: client connection to 127.4.177.5:43521: connect: Connection refused (error 111)
I20260812 06:36:33.674597  8285 raft_consensus.cc:493] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 222b20de5a1145c6b8a11df8374e8790)
I20260812 06:36:33.674702  8285 raft_consensus.cc:515] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df4308c852240859ada53d216ab69ae" member_type: VOTER last_known_addr { host: "127.4.177.6" port: 34269 } } peers { permanent_uuid: "42895a9bcba64aa484f6509605035d02" member_type: VOTER last_known_addr { host: "127.4.177.4" port: 38181 } } peers { permanent_uuid: "222b20de5a1145c6b8a11df8374e8790" member_type: VOTER last_known_addr { host: "127.4.177.5" port: 43521 } } peers { permanent_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 40897 } } peers { permanent_uuid: "8d90d61da62549208fe117b4384a469b" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 41539 } } peers { permanent_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" member_type: VOTER last_known_addr { host: "127.4.177.7" port: 46299 } } peers { permanent_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 41057 } }
I20260812 06:36:33.675320  8285 leader_election.cc:290] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 2df4308c852240859ada53d216ab69ae (127.4.177.6:34269), 42895a9bcba64aa484f6509605035d02 (127.4.177.4:38181), 222b20de5a1145c6b8a11df8374e8790 (127.4.177.5:43521), 8d90d61da62549208fe117b4384a469b (127.4.177.2:41539), b037d17b21f8422fa5e41b70eb4dcd08 (127.4.177.7:46299), ec6c8d818e744cdbb6b0d15b321010e0 (127.4.177.1:41057)
I20260812 06:36:33.675608  7099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" candidate_term: 11 candidate_status { last_received { term: 10 index: 1302 } } ignore_live_leader: false dest_uuid: "2df4308c852240859ada53d216ab69ae" is_pre_election: true
I20260812 06:36:33.675660  7320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" candidate_term: 11 candidate_status { last_received { term: 10 index: 1302 } } ignore_live_leader: false dest_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" is_pre_election: true
I20260812 06:36:33.675776  7099 raft_consensus.cc:2410] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7962f587ccd842ad8a3f6ed9a5d7707e for term 11 because replica has last-logged OpId of term: 10 index: 1307, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 1302.
I20260812 06:36:33.675755  6703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" candidate_term: 11 candidate_status { last_received { term: 10 index: 1302 } } ignore_live_leader: false dest_uuid: "42895a9bcba64aa484f6509605035d02" is_pre_election: true
I20260812 06:36:33.675787  7320 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7962f587ccd842ad8a3f6ed9a5d7707e in term 10.
I20260812 06:36:33.675895  6703 raft_consensus.cc:2410] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7962f587ccd842ad8a3f6ed9a5d7707e for term 11 because replica has last-logged OpId of term: 10 index: 1307, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 1302.
I20260812 06:36:33.676182  6249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" candidate_term: 11 candidate_status { last_received { term: 10 index: 1302 } } ignore_live_leader: false dest_uuid: "8d90d61da62549208fe117b4384a469b" is_pre_election: true
I20260812 06:36:33.676309  6249 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7962f587ccd842ad8a3f6ed9a5d7707e in term 10.
I20260812 06:36:33.676365  6046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" candidate_term: 11 candidate_status { last_received { term: 10 index: 1302 } } ignore_live_leader: false dest_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" is_pre_election: true
I20260812 06:36:33.676489  6046 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P ec6c8d818e744cdbb6b0d15b321010e0 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7962f587ccd842ad8a3f6ed9a5d7707e in term 10.
W20260812 06:36:33.676923  6369 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.4.177.5:43521: connect: Connection refused (error 111)
I20260812 06:36:33.677205  6369 leader_election.cc:304] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 6 responses out of 7 voters: 4 yes votes; 2 no votes. yes voters: 7962f587ccd842ad8a3f6ed9a5d7707e, 8d90d61da62549208fe117b4384a469b, b037d17b21f8422fa5e41b70eb4dcd08, ec6c8d818e744cdbb6b0d15b321010e0; no voters: 2df4308c852240859ada53d216ab69ae, 42895a9bcba64aa484f6509605035d02
I20260812 06:36:33.677407  8285 raft_consensus.cc:2804] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260812 06:36:33.677462  8285 raft_consensus.cc:493] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [term 10 FOLLOWER]: Starting leader election (detected failure of leader 222b20de5a1145c6b8a11df8374e8790)
I20260812 06:36:33.677522  8285 raft_consensus.cc:3060] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [term 10 FOLLOWER]: Advancing to term 11
I20260812 06:36:33.678496  8285 raft_consensus.cc:515] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df4308c852240859ada53d216ab69ae" member_type: VOTER last_known_addr { host: "127.4.177.6" port: 34269 } } peers { permanent_uuid: "42895a9bcba64aa484f6509605035d02" member_type: VOTER last_known_addr { host: "127.4.177.4" port: 38181 } } peers { permanent_uuid: "222b20de5a1145c6b8a11df8374e8790" member_type: VOTER last_known_addr { host: "127.4.177.5" port: 43521 } } peers { permanent_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 40897 } } peers { permanent_uuid: "8d90d61da62549208fe117b4384a469b" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 41539 } } peers { permanent_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" member_type: VOTER last_known_addr { host: "127.4.177.7" port: 46299 } } peers { permanent_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 41057 } }
I20260812 06:36:33.679054  8285 leader_election.cc:290] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [CANDIDATE]: Term 11 election: Requested vote from peers 2df4308c852240859ada53d216ab69ae (127.4.177.6:34269), 42895a9bcba64aa484f6509605035d02 (127.4.177.4:38181), 222b20de5a1145c6b8a11df8374e8790 (127.4.177.5:43521), 8d90d61da62549208fe117b4384a469b (127.4.177.2:41539), b037d17b21f8422fa5e41b70eb4dcd08 (127.4.177.7:46299), ec6c8d818e744cdbb6b0d15b321010e0 (127.4.177.1:41057)
I20260812 06:36:33.679284  7099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" candidate_term: 11 candidate_status { last_received { term: 10 index: 1302 } } ignore_live_leader: false dest_uuid: "2df4308c852240859ada53d216ab69ae"
I20260812 06:36:33.679418  7099 raft_consensus.cc:3060] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [term 10 FOLLOWER]: Advancing to term 11
I20260812 06:36:33.679455  6703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" candidate_term: 11 candidate_status { last_received { term: 10 index: 1302 } } ignore_live_leader: false dest_uuid: "42895a9bcba64aa484f6509605035d02"
I20260812 06:36:33.679462  7320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" candidate_term: 11 candidate_status { last_received { term: 10 index: 1302 } } ignore_live_leader: false dest_uuid: "b037d17b21f8422fa5e41b70eb4dcd08"
I20260812 06:36:33.679564  6703 raft_consensus.cc:3060] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [term 10 FOLLOWER]: Advancing to term 11
I20260812 06:36:33.679564  7320 raft_consensus.cc:3060] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08 [term 10 FOLLOWER]: Advancing to term 11
I20260812 06:36:33.680207  7099 raft_consensus.cc:2410] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 7962f587ccd842ad8a3f6ed9a5d7707e for term 11 because replica has last-logged OpId of term: 10 index: 1307, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 1302.
I20260812 06:36:33.680553  7320 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7962f587ccd842ad8a3f6ed9a5d7707e in term 11.
I20260812 06:36:33.680605  6703 raft_consensus.cc:2410] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 7962f587ccd842ad8a3f6ed9a5d7707e for term 11 because replica has last-logged OpId of term: 10 index: 1307, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 1302.
I20260812 06:36:33.680615  6249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" candidate_term: 11 candidate_status { last_received { term: 10 index: 1302 } } ignore_live_leader: false dest_uuid: "8d90d61da62549208fe117b4384a469b"
I20260812 06:36:33.680723  6249 raft_consensus.cc:3060] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b [term 10 FOLLOWER]: Advancing to term 11
I20260812 06:36:33.680814  6046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" candidate_term: 11 candidate_status { last_received { term: 10 index: 1302 } } ignore_live_leader: false dest_uuid: "ec6c8d818e744cdbb6b0d15b321010e0"
I20260812 06:36:33.680933  6046 raft_consensus.cc:3060] T 7065bb403e3047dd9a0853c49318ef1f P ec6c8d818e744cdbb6b0d15b321010e0 [term 10 FOLLOWER]: Advancing to term 11
W20260812 06:36:33.681015  6369 leader_election.cc:336] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 222b20de5a1145c6b8a11df8374e8790 (127.4.177.5:43521): Network error: Client connection negotiation failed: client connection to 127.4.177.5:43521: connect: Connection refused (error 111)
I20260812 06:36:33.681856  6046 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P ec6c8d818e744cdbb6b0d15b321010e0 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7962f587ccd842ad8a3f6ed9a5d7707e in term 11.
I20260812 06:36:33.681937  6249 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7962f587ccd842ad8a3f6ed9a5d7707e in term 11.
W20260812 06:36:33.682021  6369 leader_election.cc:336] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [CANDIDATE]: Term 11 election: RPC error from VoteRequest() call to peer 222b20de5a1145c6b8a11df8374e8790 (127.4.177.5:43521): Network error: Client connection negotiation failed: client connection to 127.4.177.5:43521: connect: Connection refused (error 111)
I20260812 06:36:33.682355  6369 leader_election.cc:304] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 7 responses out of 7 voters: 4 yes votes; 3 no votes. yes voters: 7962f587ccd842ad8a3f6ed9a5d7707e, 8d90d61da62549208fe117b4384a469b, b037d17b21f8422fa5e41b70eb4dcd08, ec6c8d818e744cdbb6b0d15b321010e0; no voters: 222b20de5a1145c6b8a11df8374e8790, 2df4308c852240859ada53d216ab69ae, 42895a9bcba64aa484f6509605035d02
I20260812 06:36:33.682551  8285 raft_consensus.cc:2804] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [term 11 FOLLOWER]: Leader election won for term 11
I20260812 06:36:33.682762  8285 raft_consensus.cc:697] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [term 11 LEADER]: Becoming Leader. State: Replica: 7962f587ccd842ad8a3f6ed9a5d7707e, State: Running, Role: LEADER
I20260812 06:36:33.682947  8285 consensus_queue.cc:237] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1297, Committed index: 1297, Last appended: 10.1302, Last appended by leader: 1302, Current term: 11, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df4308c852240859ada53d216ab69ae" member_type: VOTER last_known_addr { host: "127.4.177.6" port: 34269 } } peers { permanent_uuid: "42895a9bcba64aa484f6509605035d02" member_type: VOTER last_known_addr { host: "127.4.177.4" port: 38181 } } peers { permanent_uuid: "222b20de5a1145c6b8a11df8374e8790" member_type: VOTER last_known_addr { host: "127.4.177.5" port: 43521 } } peers { permanent_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 40897 } } peers { permanent_uuid: "8d90d61da62549208fe117b4384a469b" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 41539 } } peers { permanent_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" member_type: VOTER last_known_addr { host: "127.4.177.7" port: 46299 } } peers { permanent_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 41057 } }
I20260812 06:36:33.685249  5812 catalog_manager.cc:5719] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e reported cstate change: term changed from 10 to 11, leader changed from 222b20de5a1145c6b8a11df8374e8790 (127.4.177.5) to 7962f587ccd842ad8a3f6ed9a5d7707e (127.4.177.3). New cstate: current_term: 11 leader_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df4308c852240859ada53d216ab69ae" member_type: VOTER last_known_addr { host: "127.4.177.6" port: 34269 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "42895a9bcba64aa484f6509605035d02" member_type: VOTER last_known_addr { host: "127.4.177.4" port: 38181 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "222b20de5a1145c6b8a11df8374e8790" member_type: VOTER last_known_addr { host: "127.4.177.5" port: 43521 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 40897 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8d90d61da62549208fe117b4384a469b" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 41539 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" member_type: VOTER last_known_addr { host: "127.4.177.7" port: 46299 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 41057 } health_report { overall_health: UNKNOWN } } }
I20260812 06:36:33.722057  4804 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskCnZu03/build/debug/bin/kudu
/tmp/dist-test-taskCnZu03/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-4/wal
--fs_data_dirs=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-4/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-4/logs
--server_dump_info_path=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-4/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.177.5:43521
--local_ip_for_outbound_sockets=127.4.177.5
--tserver_master_addrs=127.4.177.62:42183
--webserver_port=43111
--webserver_interface=127.4.177.5
--builtin_ntp_servers=127.4.177.20:34335
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_cache_size_limit_mb=10
--consensus_rpc_timeout_ms=1000 with env {}
W20260812 06:36:33.836937  8298 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:36:33.837186  8298 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:36:33.837235  8298 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 06:36:33.840756  8298 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:36:33.840888  8298 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.177.5
I20260812 06:36:33.846309  8298 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-4/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.177.20:34335
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--consensus_rpc_timeout_ms=1000
--log_cache_size_limit_mb=10
--fs_data_dirs=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-4/data
--fs_wal_dir=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-4/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.177.5:43521
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-4/data/info.pb
--webserver_interface=127.4.177.5
--webserver_port=43111
--tserver_master_addrs=127.4.177.62:42183
--never_fsync=true
--heap_profile_path=/tmp/kudu.8298
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.177.5

Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 06:36:33.847564  8298 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:36:33.849292  8298 file_cache.cc:504] Constructed file cache file cache with capacity 419430
I20260812 06:36:33.858189  8298 server_base.cc:1061] running on GCE node
W20260812 06:36:33.858134  8305 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:36:33.858132  8307 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:36:33.858132  8309 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:36:33.858947  8298 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:36:33.859580  8298 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:36:33.860769  8298 hybrid_clock.cc:648] HybridClock initialized: now 1786516593860744 us; error 38 us; skew 500 ppm
I20260812 06:36:33.863231  8298 webserver.cc:533] Webserver started at http://127.4.177.5:43111/ using document root <none> and password file <none>
I20260812 06:36:33.863806  8298 fs_manager.cc:362] Metadata directory not provided
I20260812 06:36:33.863878  8298 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:36:33.867668  8298 fs_manager.cc:714] Time spent opening directory manager: real 0.003s	user 0.003s	sys 0.000s
I20260812 06:36:33.869984  8318 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 06:36:33.871786  8298 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
I20260812 06:36:33.871913  8298 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-4/data,/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-4/wal
uuid: "222b20de5a1145c6b8a11df8374e8790"
format_stamp: "Formatted at 2026-08-12 06:36:11 on dist-test-slave-jztv"
I20260812 06:36:33.872422  8298 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-4/wal
metadata directory: /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-4/wal
1 data directories: /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-4/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:36:33.896644  8298 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 06:36:33.897356  8298 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:36:33.897576  8298 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:36:33.898237  8298 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260812 06:36:33.899580  8331 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
I20260812 06:36:33.903087  8298 ts_tablet_manager.cc:585] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260812 06:36:33.903193  8298 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.004s	user 0.000s	sys 0.000s
I20260812 06:36:33.903299  8298 ts_tablet_manager.cc:600] Registering tablets (0/1 complete)
I20260812 06:36:33.905006  8298 ts_tablet_manager.cc:616] Registered 1 tablets
I20260812 06:36:33.905081  8298 ts_tablet_manager.cc:595] Time spent register tablets: real 0.002s	user 0.000s	sys 0.000s
I20260812 06:36:33.905560  8331 tablet_bootstrap.cc:492] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790: Bootstrap starting.
I20260812 06:36:33.918282  8298 rpc_server.cc:307] RPC server started. Bound to: 127.4.177.5:43521
I20260812 06:36:33.918313  8509 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.4.177.5:43521 every 8 connection(s)
I20260812 06:36:33.919524  8298 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-4/data/info.pb
I20260812 06:36:33.920691  4804 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskCnZu03/build/debug/bin/kudu as pid 8298
I20260812 06:36:33.932016  8510 heartbeater.cc:344] Connected to a master server at 127.4.177.62:42183
I20260812 06:36:33.932343  8510 heartbeater.cc:461] Registering TS with master...
I20260812 06:36:33.933066  8510 heartbeater.cc:507] Master 127.4.177.62:42183 requested a full tablet report, sending...
I20260812 06:36:33.935515  5812 ts_manager.cc:194] Re-registered known tserver with Master: 222b20de5a1145c6b8a11df8374e8790 (127.4.177.5:43521)
I20260812 06:36:33.936666  5812 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.4.177.5:47283
I20260812 06:36:33.997192  8331 log.cc:826] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790: Log is configured to *not* fsync() on all Append() calls
I20260812 06:36:34.067631  6046 raft_consensus.cc:1275] T 7065bb403e3047dd9a0853c49318ef1f P ec6c8d818e744cdbb6b0d15b321010e0 [term 11 FOLLOWER]: Refusing update from remote peer 7962f587ccd842ad8a3f6ed9a5d7707e: Log matching property violated. Preceding OpId in replica: term: 10 index: 1181. Preceding OpId from leader: term: 11 index: 1303. (index mismatch)
I20260812 06:36:34.068184  8285 consensus_queue.cc:1048] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 41057 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1303, Last known committed idx: 1178, Time since last communication: 0.000s
I20260812 06:36:34.112571  7320 raft_consensus.cc:1275] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08 [term 11 FOLLOWER]: Refusing update from remote peer 7962f587ccd842ad8a3f6ed9a5d7707e: Log matching property violated. Preceding OpId in replica: term: 10 index: 1216. Preceding OpId from leader: term: 11 index: 1303. (index mismatch)
I20260812 06:36:34.113158  8285 consensus_queue.cc:1048] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [LEADER]: Connected to new peer: Peer: permanent_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" member_type: VOTER last_known_addr { host: "127.4.177.7" port: 46299 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1303, Last known committed idx: 1213, Time since last communication: 0.000s
I20260812 06:36:34.119836  6249 raft_consensus.cc:1275] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b [term 11 FOLLOWER]: Refusing update from remote peer 7962f587ccd842ad8a3f6ed9a5d7707e: Log matching property violated. Preceding OpId in replica: term: 10 index: 1254. Preceding OpId from leader: term: 11 index: 1303. (index mismatch)
I20260812 06:36:34.122208  8285 consensus_queue.cc:1048] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8d90d61da62549208fe117b4384a469b" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 41539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1303, Last known committed idx: 1248, Time since last communication: 0.000s
I20260812 06:36:34.148433  7099 raft_consensus.cc:1275] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [term 11 FOLLOWER]: Refusing update from remote peer 7962f587ccd842ad8a3f6ed9a5d7707e: Log matching property violated. Preceding OpId in replica: term: 10 index: 1307. Preceding OpId from leader: term: 11 index: 1303. (term mismatch)
I20260812 06:36:34.148547  7099 pending_rounds.cc:85] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae: Aborting all ops after (but not including) 1302
I20260812 06:36:34.148581  7099 pending_rounds.cc:107] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae: Aborting uncommitted WRITE_OP operation due to leader change: 10.1303
I20260812 06:36:34.148855  7099 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:50492 (ReqId={client: 3067f3aef2a44cc4bb7f2ccd6acf83db, seq_no=1298, attempt_no=1}) took 1972 ms. Trace:
I20260812 06:36:34.148923  7099 rpcz_store.cc:276] 0812 06:36:32.175867 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 06:36:32.176308 (+   441us) service_pool.cc:224] Handling call
0812 06:36:34.148839 (+1972531us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260812 06:36:34.149178  7099 pending_rounds.cc:107] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae: Aborting uncommitted WRITE_OP operation due to leader change: 10.1304
I20260812 06:36:34.149369  7099 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:50492 (ReqId={client: 3067f3aef2a44cc4bb7f2ccd6acf83db, seq_no=1296, attempt_no=1}) took 1973 ms. Trace:
I20260812 06:36:34.149415  7099 rpcz_store.cc:276] 0812 06:36:32.175792 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 06:36:32.176266 (+   474us) service_pool.cc:224] Handling call
0812 06:36:34.149359 (+1973093us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260812 06:36:34.149580  7099 pending_rounds.cc:107] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae: Aborting uncommitted WRITE_OP operation due to leader change: 10.1305
I20260812 06:36:34.149731  7099 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:50492 (ReqId={client: 3067f3aef2a44cc4bb7f2ccd6acf83db, seq_no=1297, attempt_no=1}) took 1973 ms. Trace:
I20260812 06:36:34.149780  7099 rpcz_store.cc:276] 0812 06:36:32.175903 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 06:36:32.176137 (+   234us) service_pool.cc:224] Handling call
0812 06:36:34.149721 (+1973584us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260812 06:36:34.149919  7099 pending_rounds.cc:107] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae: Aborting uncommitted WRITE_OP operation due to leader change: 10.1306
I20260812 06:36:34.150055  7099 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:50492 (ReqId={client: 3067f3aef2a44cc4bb7f2ccd6acf83db, seq_no=1299, attempt_no=1}) took 1974 ms. Trace:
I20260812 06:36:34.150097  7099 rpcz_store.cc:276] 0812 06:36:32.175753 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 06:36:32.176400 (+   647us) service_pool.cc:224] Handling call
0812 06:36:34.150047 (+1973647us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260812 06:36:34.150254  7099 pending_rounds.cc:107] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae: Aborting uncommitted WRITE_OP operation due to leader change: 10.1307
I20260812 06:36:34.150424  7099 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:50492 (ReqId={client: 3067f3aef2a44cc4bb7f2ccd6acf83db, seq_no=1300, attempt_no=1}) took 1974 ms. Trace:
I20260812 06:36:34.150475  7099 rpcz_store.cc:276] 0812 06:36:32.175700 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 06:36:32.176064 (+   364us) service_pool.cc:224] Handling call
0812 06:36:34.150415 (+1974351us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260812 06:36:34.151057  6703 raft_consensus.cc:1275] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [term 11 FOLLOWER]: Refusing update from remote peer 7962f587ccd842ad8a3f6ed9a5d7707e: Log matching property violated. Preceding OpId in replica: term: 10 index: 1307. Preceding OpId from leader: term: 11 index: 1303. (term mismatch)
I20260812 06:36:34.151139  6703 pending_rounds.cc:85] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02: Aborting all ops after (but not including) 1302
I20260812 06:36:34.151162  6703 pending_rounds.cc:107] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02: Aborting uncommitted WRITE_OP operation due to leader change: 10.1303
I20260812 06:36:34.151399  6703 pending_rounds.cc:107] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02: Aborting uncommitted WRITE_OP operation due to leader change: 10.1304
I20260812 06:36:34.151593  6703 pending_rounds.cc:107] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02: Aborting uncommitted WRITE_OP operation due to leader change: 10.1305
I20260812 06:36:34.151757  6703 pending_rounds.cc:107] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02: Aborting uncommitted WRITE_OP operation due to leader change: 10.1306
I20260812 06:36:34.151898  6703 pending_rounds.cc:107] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02: Aborting uncommitted WRITE_OP operation due to leader change: 10.1307
W20260812 06:36:34.153841  7063 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50492: Illegal state: replica 2df4308c852240859ada53d216ab69ae is not leader of this config: current role FOLLOWER
I20260812 06:36:34.156167  8290 consensus_queue.cc:1048] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [LEADER]: Connected to new peer: Peer: permanent_uuid: "42895a9bcba64aa484f6509605035d02" member_type: VOTER last_known_addr { host: "127.4.177.4" port: 38181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1303, Last known committed idx: 1299, Time since last communication: 0.000s
I20260812 06:36:34.156507  8290 consensus_queue.cc:1048] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [LEADER]: Connected to new peer: Peer: permanent_uuid: "2df4308c852240859ada53d216ab69ae" member_type: VOTER last_known_addr { host: "127.4.177.6" port: 34269 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1303, Last known committed idx: 1299, Time since last communication: 0.000s
W20260812 06:36:34.158479  7061 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50492: Illegal state: replica 2df4308c852240859ada53d216ab69ae is not leader of this config: current role FOLLOWER
W20260812 06:36:34.159099  7063 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50492: Illegal state: replica 2df4308c852240859ada53d216ab69ae is not leader of this config: current role FOLLOWER
W20260812 06:36:34.159806  6621 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:42408: Illegal state: replica 42895a9bcba64aa484f6509605035d02 is not leader of this config: current role FOLLOWER
W20260812 06:36:34.164537  6369 consensus_peers.cc:597] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e -> Peer 222b20de5a1145c6b8a11df8374e8790 (127.4.177.5:43521): Couldn't send request to peer 222b20de5a1145c6b8a11df8374e8790. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
W20260812 06:36:34.165791  6621 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:42408: Illegal state: replica 42895a9bcba64aa484f6509605035d02 is not leader of this config: current role FOLLOWER
W20260812 06:36:34.166466  6626 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:42408: Illegal state: replica 42895a9bcba64aa484f6509605035d02 is not leader of this config: current role FOLLOWER
I20260812 06:36:34.188782  8599 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:50492 (ReqId={client: 3067f3aef2a44cc4bb7f2ccd6acf83db, seq_no=1293, attempt_no=1}) took 2012 ms. Trace:
I20260812 06:36:34.188999  8599 rpcz_store.cc:276] 0812 06:36:32.175976 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 06:36:32.176440 (+   464us) service_pool.cc:224] Handling call
0812 06:36:34.188763 (+2012323us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260812 06:36:34.191587  8598 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:50492 (ReqId={client: 3067f3aef2a44cc4bb7f2ccd6acf83db, seq_no=1295, attempt_no=1}) took 2015 ms. Trace:
I20260812 06:36:34.191736  8598 rpcz_store.cc:276] 0812 06:36:32.175830 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 06:36:32.176222 (+   392us) service_pool.cc:224] Handling call
0812 06:36:34.191557 (+2015335us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260812 06:36:34.193523  8599 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:50492 (ReqId={client: 3067f3aef2a44cc4bb7f2ccd6acf83db, seq_no=1294, attempt_no=1}) took 2017 ms. Trace:
I20260812 06:36:34.193611  8599 rpcz_store.cc:276] 0812 06:36:32.175939 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 06:36:32.176357 (+   418us) service_pool.cc:224] Handling call
0812 06:36:34.193486 (+2017129us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260812 06:36:35.174125  4804 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskCnZu03/build/debug/bin/kudu with pid 6333
W20260812 06:36:35.189321  7401 meta_cache.cc:302] tablet 7065bb403e3047dd9a0853c49318ef1f: replica 7962f587ccd842ad8a3f6ed9a5d7707e (127.4.177.3:40897) has failed: Network error: recv got EOF from 127.4.177.3:40897 (error 108)
I20260812 06:36:35.191977  8510 heartbeater.cc:499] Master 127.4.177.62:42183 was elected leader, sending a full tablet report...
W20260812 06:36:35.197180  6147 connection.cc:466] server connection from 127.4.177.3:50207 torn down before Call kudu.consensus.ConsensusService.UpdateConsensus from 127.4.177.3:50207 (request call id 242) could send its response
W20260812 06:36:35.201828  5939 connection.cc:466] server connection from 127.4.177.3:34723 torn down before Call kudu.consensus.ConsensusService.UpdateConsensus from 127.4.177.3:34723 (request call id 153) could send its response
I20260812 06:36:35.497804  8331 tablet_bootstrap.cc:492] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790: Bootstrap replayed 1/1 log segments. Stats: ops{read=1312 overwritten=5 applied=1299 ignored=0} inserts{seen=12930 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 8 replicates
I20260812 06:36:35.498406  8331 tablet_bootstrap.cc:492] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790: Bootstrap complete.
I20260812 06:36:35.499614  8331 ts_tablet_manager.cc:1403] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790: Time spent bootstrapping tablet: real 1.594s	user 1.079s	sys 0.068s
I20260812 06:36:35.501912  8331 raft_consensus.cc:359] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [term 10 FOLLOWER]: Replica starting. Triggering 8 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df4308c852240859ada53d216ab69ae" member_type: VOTER last_known_addr { host: "127.4.177.6" port: 34269 } } peers { permanent_uuid: "42895a9bcba64aa484f6509605035d02" member_type: VOTER last_known_addr { host: "127.4.177.4" port: 38181 } } peers { permanent_uuid: "222b20de5a1145c6b8a11df8374e8790" member_type: VOTER last_known_addr { host: "127.4.177.5" port: 43521 } } peers { permanent_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 40897 } } peers { permanent_uuid: "8d90d61da62549208fe117b4384a469b" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 41539 } } peers { permanent_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" member_type: VOTER last_known_addr { host: "127.4.177.7" port: 46299 } } peers { permanent_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 41057 } }
I20260812 06:36:35.503952  8331 raft_consensus.cc:740] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [term 10 FOLLOWER]: Becoming Follower/Learner. State: Replica: 222b20de5a1145c6b8a11df8374e8790, State: Initialized, Role: FOLLOWER
I20260812 06:36:35.504505  8331 consensus_queue.cc:260] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1299, Last appended: 10.1307, Last appended by leader: 1307, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df4308c852240859ada53d216ab69ae" member_type: VOTER last_known_addr { host: "127.4.177.6" port: 34269 } } peers { permanent_uuid: "42895a9bcba64aa484f6509605035d02" member_type: VOTER last_known_addr { host: "127.4.177.4" port: 38181 } } peers { permanent_uuid: "222b20de5a1145c6b8a11df8374e8790" member_type: VOTER last_known_addr { host: "127.4.177.5" port: 43521 } } peers { permanent_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 40897 } } peers { permanent_uuid: "8d90d61da62549208fe117b4384a469b" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 41539 } } peers { permanent_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" member_type: VOTER last_known_addr { host: "127.4.177.7" port: 46299 } } peers { permanent_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 41057 } }
I20260812 06:36:35.505383  8331 ts_tablet_manager.cc:1434] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790: Time spent starting tablet: real 0.006s	user 0.002s	sys 0.004s
I20260812 06:36:36.008059  8618 raft_consensus.cc:493] T 7065bb403e3047dd9a0853c49318ef1f P ec6c8d818e744cdbb6b0d15b321010e0 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 7962f587ccd842ad8a3f6ed9a5d7707e)
I20260812 06:36:36.008195  8618 raft_consensus.cc:515] T 7065bb403e3047dd9a0853c49318ef1f P ec6c8d818e744cdbb6b0d15b321010e0 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df4308c852240859ada53d216ab69ae" member_type: VOTER last_known_addr { host: "127.4.177.6" port: 34269 } } peers { permanent_uuid: "42895a9bcba64aa484f6509605035d02" member_type: VOTER last_known_addr { host: "127.4.177.4" port: 38181 } } peers { permanent_uuid: "222b20de5a1145c6b8a11df8374e8790" member_type: VOTER last_known_addr { host: "127.4.177.5" port: 43521 } } peers { permanent_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 40897 } } peers { permanent_uuid: "8d90d61da62549208fe117b4384a469b" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 41539 } } peers { permanent_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" member_type: VOTER last_known_addr { host: "127.4.177.7" port: 46299 } } peers { permanent_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 41057 } }
I20260812 06:36:36.009075  8618 leader_election.cc:290] T 7065bb403e3047dd9a0853c49318ef1f P ec6c8d818e744cdbb6b0d15b321010e0 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 2df4308c852240859ada53d216ab69ae (127.4.177.6:34269), 42895a9bcba64aa484f6509605035d02 (127.4.177.4:38181), 222b20de5a1145c6b8a11df8374e8790 (127.4.177.5:43521), 7962f587ccd842ad8a3f6ed9a5d7707e (127.4.177.3:40897), 8d90d61da62549208fe117b4384a469b (127.4.177.2:41539), b037d17b21f8422fa5e41b70eb4dcd08 (127.4.177.7:46299)
I20260812 06:36:36.009228  6703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" candidate_term: 12 candidate_status { last_received { term: 11 index: 1366 } } ignore_live_leader: false dest_uuid: "42895a9bcba64aa484f6509605035d02" is_pre_election: true
I20260812 06:36:36.009446  7099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" candidate_term: 12 candidate_status { last_received { term: 11 index: 1366 } } ignore_live_leader: false dest_uuid: "2df4308c852240859ada53d216ab69ae" is_pre_election: true
I20260812 06:36:36.009740  7321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" candidate_term: 12 candidate_status { last_received { term: 11 index: 1366 } } ignore_live_leader: false dest_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" is_pre_election: true
I20260812 06:36:36.010200  6249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" candidate_term: 12 candidate_status { last_received { term: 11 index: 1366 } } ignore_live_leader: false dest_uuid: "8d90d61da62549208fe117b4384a469b" is_pre_election: true
I20260812 06:36:36.010659  5945 leader_election.cc:304] T 7065bb403e3047dd9a0853c49318ef1f P ec6c8d818e744cdbb6b0d15b321010e0 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 5 responses out of 7 voters: 1 yes votes; 4 no votes. yes voters: ec6c8d818e744cdbb6b0d15b321010e0; no voters: 2df4308c852240859ada53d216ab69ae, 42895a9bcba64aa484f6509605035d02, 8d90d61da62549208fe117b4384a469b, b037d17b21f8422fa5e41b70eb4dcd08
I20260812 06:36:36.010841  8618 raft_consensus.cc:2749] T 7065bb403e3047dd9a0853c49318ef1f P ec6c8d818e744cdbb6b0d15b321010e0 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260812 06:36:36.011428  5945 leader_election.cc:336] T 7065bb403e3047dd9a0853c49318ef1f P ec6c8d818e744cdbb6b0d15b321010e0 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 7962f587ccd842ad8a3f6ed9a5d7707e (127.4.177.3:40897): Network error: Client connection negotiation failed: client connection to 127.4.177.3:40897: connect: Connection refused (error 111)
I20260812 06:36:36.013573  8439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" candidate_term: 12 candidate_status { last_received { term: 11 index: 1366 } } ignore_live_leader: false dest_uuid: "222b20de5a1145c6b8a11df8374e8790" is_pre_election: true
I20260812 06:36:36.013926  8439 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec6c8d818e744cdbb6b0d15b321010e0 in term 10.
I20260812 06:36:36.107688  8625 raft_consensus.cc:493] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 7962f587ccd842ad8a3f6ed9a5d7707e)
I20260812 06:36:36.107837  8625 raft_consensus.cc:515] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df4308c852240859ada53d216ab69ae" member_type: VOTER last_known_addr { host: "127.4.177.6" port: 34269 } } peers { permanent_uuid: "42895a9bcba64aa484f6509605035d02" member_type: VOTER last_known_addr { host: "127.4.177.4" port: 38181 } } peers { permanent_uuid: "222b20de5a1145c6b8a11df8374e8790" member_type: VOTER last_known_addr { host: "127.4.177.5" port: 43521 } } peers { permanent_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 40897 } } peers { permanent_uuid: "8d90d61da62549208fe117b4384a469b" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 41539 } } peers { permanent_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" member_type: VOTER last_known_addr { host: "127.4.177.7" port: 46299 } } peers { permanent_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 41057 } }
I20260812 06:36:36.108673  8625 leader_election.cc:290] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 2df4308c852240859ada53d216ab69ae (127.4.177.6:34269), 42895a9bcba64aa484f6509605035d02 (127.4.177.4:38181), 222b20de5a1145c6b8a11df8374e8790 (127.4.177.5:43521), 7962f587ccd842ad8a3f6ed9a5d7707e (127.4.177.3:40897), 8d90d61da62549208fe117b4384a469b (127.4.177.2:41539), ec6c8d818e744cdbb6b0d15b321010e0 (127.4.177.1:41057)
I20260812 06:36:36.109107  6703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" candidate_term: 12 candidate_status { last_received { term: 11 index: 1391 } } ignore_live_leader: false dest_uuid: "42895a9bcba64aa484f6509605035d02" is_pre_election: true
I20260812 06:36:36.109107  7099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" candidate_term: 12 candidate_status { last_received { term: 11 index: 1391 } } ignore_live_leader: false dest_uuid: "2df4308c852240859ada53d216ab69ae" is_pre_election: true
I20260812 06:36:36.109977  6249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" candidate_term: 12 candidate_status { last_received { term: 11 index: 1391 } } ignore_live_leader: false dest_uuid: "8d90d61da62549208fe117b4384a469b" is_pre_election: true
I20260812 06:36:36.110383  6046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" candidate_term: 12 candidate_status { last_received { term: 11 index: 1391 } } ignore_live_leader: false dest_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" is_pre_election: true
I20260812 06:36:36.110649  6046 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P ec6c8d818e744cdbb6b0d15b321010e0 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b037d17b21f8422fa5e41b70eb4dcd08 in term 11.
W20260812 06:36:36.111166  7211 leader_election.cc:336] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 7962f587ccd842ad8a3f6ed9a5d7707e (127.4.177.3:40897): Network error: Client connection negotiation failed: client connection to 127.4.177.3:40897: connect: Connection refused (error 111)
I20260812 06:36:36.111236  7211 leader_election.cc:304] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 6 responses out of 7 voters: 2 yes votes; 4 no votes. yes voters: b037d17b21f8422fa5e41b70eb4dcd08, ec6c8d818e744cdbb6b0d15b321010e0; no voters: 2df4308c852240859ada53d216ab69ae, 42895a9bcba64aa484f6509605035d02, 7962f587ccd842ad8a3f6ed9a5d7707e, 8d90d61da62549208fe117b4384a469b
I20260812 06:36:36.111384  8625 raft_consensus.cc:2749] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260812 06:36:36.113138  8439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" candidate_term: 12 candidate_status { last_received { term: 11 index: 1391 } } ignore_live_leader: false dest_uuid: "222b20de5a1145c6b8a11df8374e8790" is_pre_election: true
I20260812 06:36:36.113272  8439 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b037d17b21f8422fa5e41b70eb4dcd08 in term 10.
I20260812 06:36:36.230502  8631 raft_consensus.cc:493] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 7962f587ccd842ad8a3f6ed9a5d7707e)
I20260812 06:36:36.230651  8631 raft_consensus.cc:515] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df4308c852240859ada53d216ab69ae" member_type: VOTER last_known_addr { host: "127.4.177.6" port: 34269 } } peers { permanent_uuid: "42895a9bcba64aa484f6509605035d02" member_type: VOTER last_known_addr { host: "127.4.177.4" port: 38181 } } peers { permanent_uuid: "222b20de5a1145c6b8a11df8374e8790" member_type: VOTER last_known_addr { host: "127.4.177.5" port: 43521 } } peers { permanent_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 40897 } } peers { permanent_uuid: "8d90d61da62549208fe117b4384a469b" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 41539 } } peers { permanent_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" member_type: VOTER last_known_addr { host: "127.4.177.7" port: 46299 } } peers { permanent_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 41057 } }
I20260812 06:36:36.231328  8631 leader_election.cc:290] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 2df4308c852240859ada53d216ab69ae (127.4.177.6:34269), 42895a9bcba64aa484f6509605035d02 (127.4.177.4:38181), 222b20de5a1145c6b8a11df8374e8790 (127.4.177.5:43521), 7962f587ccd842ad8a3f6ed9a5d7707e (127.4.177.3:40897), b037d17b21f8422fa5e41b70eb4dcd08 (127.4.177.7:46299), ec6c8d818e744cdbb6b0d15b321010e0 (127.4.177.1:41057)
I20260812 06:36:36.231730  7099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "8d90d61da62549208fe117b4384a469b" candidate_term: 12 candidate_status { last_received { term: 11 index: 1430 } } ignore_live_leader: false dest_uuid: "2df4308c852240859ada53d216ab69ae" is_pre_election: true
I20260812 06:36:36.231824  6703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "8d90d61da62549208fe117b4384a469b" candidate_term: 12 candidate_status { last_received { term: 11 index: 1430 } } ignore_live_leader: false dest_uuid: "42895a9bcba64aa484f6509605035d02" is_pre_election: true
I20260812 06:36:36.232223  7321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "8d90d61da62549208fe117b4384a469b" candidate_term: 12 candidate_status { last_received { term: 11 index: 1430 } } ignore_live_leader: false dest_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" is_pre_election: true
I20260812 06:36:36.232376  7321 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8d90d61da62549208fe117b4384a469b in term 11.
I20260812 06:36:36.232733  6046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "8d90d61da62549208fe117b4384a469b" candidate_term: 12 candidate_status { last_received { term: 11 index: 1430 } } ignore_live_leader: false dest_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" is_pre_election: true
I20260812 06:36:36.232887  6046 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P ec6c8d818e744cdbb6b0d15b321010e0 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8d90d61da62549208fe117b4384a469b in term 11.
W20260812 06:36:36.233378  6150 leader_election.cc:336] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 7962f587ccd842ad8a3f6ed9a5d7707e (127.4.177.3:40897): Network error: Client connection negotiation failed: client connection to 127.4.177.3:40897: connect: Connection refused (error 111)
I20260812 06:36:36.235733  8439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "8d90d61da62549208fe117b4384a469b" candidate_term: 12 candidate_status { last_received { term: 11 index: 1430 } } ignore_live_leader: false dest_uuid: "222b20de5a1145c6b8a11df8374e8790" is_pre_election: true
I20260812 06:36:36.235877  8439 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8d90d61da62549208fe117b4384a469b in term 10.
I20260812 06:36:36.236184  6150 leader_election.cc:304] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 7 responses out of 7 voters: 4 yes votes; 3 no votes. yes voters: 222b20de5a1145c6b8a11df8374e8790, 8d90d61da62549208fe117b4384a469b, b037d17b21f8422fa5e41b70eb4dcd08, ec6c8d818e744cdbb6b0d15b321010e0; no voters: 2df4308c852240859ada53d216ab69ae, 42895a9bcba64aa484f6509605035d02, 7962f587ccd842ad8a3f6ed9a5d7707e
I20260812 06:36:36.236390  8631 raft_consensus.cc:2804] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260812 06:36:36.236470  8631 raft_consensus.cc:493] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b [term 11 FOLLOWER]: Starting leader election (detected failure of leader 7962f587ccd842ad8a3f6ed9a5d7707e)
I20260812 06:36:36.236529  8631 raft_consensus.cc:3060] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b [term 11 FOLLOWER]: Advancing to term 12
I20260812 06:36:36.237423  8631 raft_consensus.cc:515] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df4308c852240859ada53d216ab69ae" member_type: VOTER last_known_addr { host: "127.4.177.6" port: 34269 } } peers { permanent_uuid: "42895a9bcba64aa484f6509605035d02" member_type: VOTER last_known_addr { host: "127.4.177.4" port: 38181 } } peers { permanent_uuid: "222b20de5a1145c6b8a11df8374e8790" member_type: VOTER last_known_addr { host: "127.4.177.5" port: 43521 } } peers { permanent_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 40897 } } peers { permanent_uuid: "8d90d61da62549208fe117b4384a469b" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 41539 } } peers { permanent_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" member_type: VOTER last_known_addr { host: "127.4.177.7" port: 46299 } } peers { permanent_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 41057 } }
I20260812 06:36:36.237946  8631 leader_election.cc:290] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b [CANDIDATE]: Term 12 election: Requested vote from peers 2df4308c852240859ada53d216ab69ae (127.4.177.6:34269), 42895a9bcba64aa484f6509605035d02 (127.4.177.4:38181), 222b20de5a1145c6b8a11df8374e8790 (127.4.177.5:43521), 7962f587ccd842ad8a3f6ed9a5d7707e (127.4.177.3:40897), b037d17b21f8422fa5e41b70eb4dcd08 (127.4.177.7:46299), ec6c8d818e744cdbb6b0d15b321010e0 (127.4.177.1:41057)
I20260812 06:36:36.238284  7099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "8d90d61da62549208fe117b4384a469b" candidate_term: 12 candidate_status { last_received { term: 11 index: 1430 } } ignore_live_leader: false dest_uuid: "2df4308c852240859ada53d216ab69ae"
I20260812 06:36:36.238359  6703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "8d90d61da62549208fe117b4384a469b" candidate_term: 12 candidate_status { last_received { term: 11 index: 1430 } } ignore_live_leader: false dest_uuid: "42895a9bcba64aa484f6509605035d02"
I20260812 06:36:36.238360  7321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "8d90d61da62549208fe117b4384a469b" candidate_term: 12 candidate_status { last_received { term: 11 index: 1430 } } ignore_live_leader: false dest_uuid: "b037d17b21f8422fa5e41b70eb4dcd08"
I20260812 06:36:36.238530  7321 raft_consensus.cc:3060] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08 [term 11 FOLLOWER]: Advancing to term 12
I20260812 06:36:36.238934  8439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "8d90d61da62549208fe117b4384a469b" candidate_term: 12 candidate_status { last_received { term: 11 index: 1430 } } ignore_live_leader: false dest_uuid: "222b20de5a1145c6b8a11df8374e8790"
I20260812 06:36:36.239079  8439 raft_consensus.cc:3060] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [term 10 FOLLOWER]: Advancing to term 12
I20260812 06:36:36.239511  6046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "8d90d61da62549208fe117b4384a469b" candidate_term: 12 candidate_status { last_received { term: 11 index: 1430 } } ignore_live_leader: false dest_uuid: "ec6c8d818e744cdbb6b0d15b321010e0"
I20260812 06:36:36.239646  6046 raft_consensus.cc:3060] T 7065bb403e3047dd9a0853c49318ef1f P ec6c8d818e744cdbb6b0d15b321010e0 [term 11 FOLLOWER]: Advancing to term 12
I20260812 06:36:36.239741  7321 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8d90d61da62549208fe117b4384a469b in term 12.
W20260812 06:36:36.240710  6150 leader_election.cc:336] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b [CANDIDATE]: Term 12 election: RPC error from VoteRequest() call to peer 7962f587ccd842ad8a3f6ed9a5d7707e (127.4.177.3:40897): Network error: Client connection negotiation failed: client connection to 127.4.177.3:40897: connect: Connection refused (error 111)
I20260812 06:36:36.240708  6046 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P ec6c8d818e744cdbb6b0d15b321010e0 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8d90d61da62549208fe117b4384a469b in term 12.
I20260812 06:36:36.241408  8439 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8d90d61da62549208fe117b4384a469b in term 12.
I20260812 06:36:36.241788  6150 leader_election.cc:304] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 7 responses out of 7 voters: 4 yes votes; 3 no votes. yes voters: 222b20de5a1145c6b8a11df8374e8790, 8d90d61da62549208fe117b4384a469b, b037d17b21f8422fa5e41b70eb4dcd08, ec6c8d818e744cdbb6b0d15b321010e0; no voters: 2df4308c852240859ada53d216ab69ae, 42895a9bcba64aa484f6509605035d02, 7962f587ccd842ad8a3f6ed9a5d7707e
I20260812 06:36:36.242043  8631 raft_consensus.cc:2804] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b [term 12 FOLLOWER]: Leader election won for term 12
I20260812 06:36:36.242324  8631 raft_consensus.cc:697] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b [term 12 LEADER]: Becoming Leader. State: Replica: 8d90d61da62549208fe117b4384a469b, State: Running, Role: LEADER
I20260812 06:36:36.242692  8631 consensus_queue.cc:237] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1427, Committed index: 1427, Last appended: 11.1430, Last appended by leader: 1430, Current term: 12, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df4308c852240859ada53d216ab69ae" member_type: VOTER last_known_addr { host: "127.4.177.6" port: 34269 } } peers { permanent_uuid: "42895a9bcba64aa484f6509605035d02" member_type: VOTER last_known_addr { host: "127.4.177.4" port: 38181 } } peers { permanent_uuid: "222b20de5a1145c6b8a11df8374e8790" member_type: VOTER last_known_addr { host: "127.4.177.5" port: 43521 } } peers { permanent_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 40897 } } peers { permanent_uuid: "8d90d61da62549208fe117b4384a469b" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 41539 } } peers { permanent_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" member_type: VOTER last_known_addr { host: "127.4.177.7" port: 46299 } } peers { permanent_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 41057 } }
I20260812 06:36:36.244905  4804 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskCnZu03/build/debug/bin/kudu
/tmp/dist-test-taskCnZu03/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.177.3:40897
--local_ip_for_outbound_sockets=127.4.177.3
--tserver_master_addrs=127.4.177.62:42183
--webserver_port=44527
--webserver_interface=127.4.177.3
--builtin_ntp_servers=127.4.177.20:34335
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_cache_size_limit_mb=10
--consensus_rpc_timeout_ms=1000 with env {}
I20260812 06:36:36.245563  5812 catalog_manager.cc:5719] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b reported cstate change: term changed from 11 to 12, leader changed from 7962f587ccd842ad8a3f6ed9a5d7707e (127.4.177.3) to 8d90d61da62549208fe117b4384a469b (127.4.177.2). New cstate: current_term: 12 leader_uuid: "8d90d61da62549208fe117b4384a469b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df4308c852240859ada53d216ab69ae" member_type: VOTER last_known_addr { host: "127.4.177.6" port: 34269 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "42895a9bcba64aa484f6509605035d02" member_type: VOTER last_known_addr { host: "127.4.177.4" port: 38181 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "222b20de5a1145c6b8a11df8374e8790" member_type: VOTER last_known_addr { host: "127.4.177.5" port: 43521 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 40897 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8d90d61da62549208fe117b4384a469b" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 41539 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" member_type: VOTER last_known_addr { host: "127.4.177.7" port: 46299 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 41057 } health_report { overall_health: UNKNOWN } } }
I20260812 06:36:36.250288  8645 raft_consensus.cc:493] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 7962f587ccd842ad8a3f6ed9a5d7707e)
I20260812 06:36:36.250423  8645 raft_consensus.cc:515] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df4308c852240859ada53d216ab69ae" member_type: VOTER last_known_addr { host: "127.4.177.6" port: 34269 } } peers { permanent_uuid: "42895a9bcba64aa484f6509605035d02" member_type: VOTER last_known_addr { host: "127.4.177.4" port: 38181 } } peers { permanent_uuid: "222b20de5a1145c6b8a11df8374e8790" member_type: VOTER last_known_addr { host: "127.4.177.5" port: 43521 } } peers { permanent_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 40897 } } peers { permanent_uuid: "8d90d61da62549208fe117b4384a469b" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 41539 } } peers { permanent_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" member_type: VOTER last_known_addr { host: "127.4.177.7" port: 46299 } } peers { permanent_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 41057 } }
I20260812 06:36:36.250512  8649 raft_consensus.cc:493] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 7962f587ccd842ad8a3f6ed9a5d7707e)
I20260812 06:36:36.250609  8649 raft_consensus.cc:515] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df4308c852240859ada53d216ab69ae" member_type: VOTER last_known_addr { host: "127.4.177.6" port: 34269 } } peers { permanent_uuid: "42895a9bcba64aa484f6509605035d02" member_type: VOTER last_known_addr { host: "127.4.177.4" port: 38181 } } peers { permanent_uuid: "222b20de5a1145c6b8a11df8374e8790" member_type: VOTER last_known_addr { host: "127.4.177.5" port: 43521 } } peers { permanent_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 40897 } } peers { permanent_uuid: "8d90d61da62549208fe117b4384a469b" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 41539 } } peers { permanent_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" member_type: VOTER last_known_addr { host: "127.4.177.7" port: 46299 } } peers { permanent_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 41057 } }
I20260812 06:36:36.251173  8645 leader_election.cc:290] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 2df4308c852240859ada53d216ab69ae (127.4.177.6:34269), 222b20de5a1145c6b8a11df8374e8790 (127.4.177.5:43521), 7962f587ccd842ad8a3f6ed9a5d7707e (127.4.177.3:40897), 8d90d61da62549208fe117b4384a469b (127.4.177.2:41539), b037d17b21f8422fa5e41b70eb4dcd08 (127.4.177.7:46299), ec6c8d818e744cdbb6b0d15b321010e0 (127.4.177.1:41057)
I20260812 06:36:36.251543  6703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "2df4308c852240859ada53d216ab69ae" candidate_term: 12 candidate_status { last_received { term: 11 index: 1438 } } ignore_live_leader: false dest_uuid: "42895a9bcba64aa484f6509605035d02" is_pre_election: true
I20260812 06:36:36.251662  6703 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2df4308c852240859ada53d216ab69ae in term 11.
I20260812 06:36:36.251708  7321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "42895a9bcba64aa484f6509605035d02" candidate_term: 12 candidate_status { last_received { term: 11 index: 1438 } } ignore_live_leader: false dest_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" is_pre_election: true
I20260812 06:36:36.251863  7321 raft_consensus.cc:2393] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 42895a9bcba64aa484f6509605035d02 in current term 12: Already voted for candidate 8d90d61da62549208fe117b4384a469b in this term.
I20260812 06:36:36.252413  6046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "42895a9bcba64aa484f6509605035d02" candidate_term: 12 candidate_status { last_received { term: 11 index: 1438 } } ignore_live_leader: false dest_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" is_pre_election: true
I20260812 06:36:36.252542  7099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "42895a9bcba64aa484f6509605035d02" candidate_term: 12 candidate_status { last_received { term: 11 index: 1438 } } ignore_live_leader: false dest_uuid: "2df4308c852240859ada53d216ab69ae" is_pre_election: true
I20260812 06:36:36.252612  6046 raft_consensus.cc:2393] T 7065bb403e3047dd9a0853c49318ef1f P ec6c8d818e744cdbb6b0d15b321010e0 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 42895a9bcba64aa484f6509605035d02 in current term 12: Already voted for candidate 8d90d61da62549208fe117b4384a469b in this term.
I20260812 06:36:36.252671  7099 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 42895a9bcba64aa484f6509605035d02 in term 11.
I20260812 06:36:36.252852  8649 leader_election.cc:290] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 42895a9bcba64aa484f6509605035d02 (127.4.177.4:38181), 222b20de5a1145c6b8a11df8374e8790 (127.4.177.5:43521), 7962f587ccd842ad8a3f6ed9a5d7707e (127.4.177.3:40897), 8d90d61da62549208fe117b4384a469b (127.4.177.2:41539), b037d17b21f8422fa5e41b70eb4dcd08 (127.4.177.7:46299), ec6c8d818e744cdbb6b0d15b321010e0 (127.4.177.1:41057)
I20260812 06:36:36.252892  6249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "42895a9bcba64aa484f6509605035d02" candidate_term: 12 candidate_status { last_received { term: 11 index: 1438 } } ignore_live_leader: false dest_uuid: "8d90d61da62549208fe117b4384a469b" is_pre_election: true
I20260812 06:36:36.253424  6046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "2df4308c852240859ada53d216ab69ae" candidate_term: 12 candidate_status { last_received { term: 11 index: 1438 } } ignore_live_leader: false dest_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" is_pre_election: true
I20260812 06:36:36.253553  6046 raft_consensus.cc:2393] T 7065bb403e3047dd9a0853c49318ef1f P ec6c8d818e744cdbb6b0d15b321010e0 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2df4308c852240859ada53d216ab69ae in current term 12: Already voted for candidate 8d90d61da62549208fe117b4384a469b in this term.
I20260812 06:36:36.254146  6249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "2df4308c852240859ada53d216ab69ae" candidate_term: 12 candidate_status { last_received { term: 11 index: 1438 } } ignore_live_leader: false dest_uuid: "8d90d61da62549208fe117b4384a469b" is_pre_election: true
I20260812 06:36:36.254365  7321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "2df4308c852240859ada53d216ab69ae" candidate_term: 12 candidate_status { last_received { term: 11 index: 1438 } } ignore_live_leader: false dest_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" is_pre_election: true
I20260812 06:36:36.254527  7321 raft_consensus.cc:2393] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2df4308c852240859ada53d216ab69ae in current term 12: Already voted for candidate 8d90d61da62549208fe117b4384a469b in this term.
W20260812 06:36:36.256234  7020 leader_election.cc:336] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 7962f587ccd842ad8a3f6ed9a5d7707e (127.4.177.3:40897): Network error: Client connection negotiation failed: client connection to 127.4.177.3:40897: connect: Connection refused (error 111)
I20260812 06:36:36.256316  7020 leader_election.cc:304] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 6 responses out of 7 voters: 2 yes votes; 4 no votes. yes voters: 2df4308c852240859ada53d216ab69ae, 42895a9bcba64aa484f6509605035d02; no voters: 7962f587ccd842ad8a3f6ed9a5d7707e, 8d90d61da62549208fe117b4384a469b, b037d17b21f8422fa5e41b70eb4dcd08, ec6c8d818e744cdbb6b0d15b321010e0
I20260812 06:36:36.256480  8649 raft_consensus.cc:3060] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [term 11 FOLLOWER]: Advancing to term 12
W20260812 06:36:36.257447  6597 leader_election.cc:336] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 7962f587ccd842ad8a3f6ed9a5d7707e (127.4.177.3:40897): Network error: Client connection negotiation failed: client connection to 127.4.177.3:40897: connect: Connection refused (error 111)
I20260812 06:36:36.257424  8439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "42895a9bcba64aa484f6509605035d02" candidate_term: 12 candidate_status { last_received { term: 11 index: 1438 } } ignore_live_leader: false dest_uuid: "222b20de5a1145c6b8a11df8374e8790" is_pre_election: true
I20260812 06:36:36.257519  6597 leader_election.cc:304] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 6 responses out of 7 voters: 2 yes votes; 4 no votes. yes voters: 2df4308c852240859ada53d216ab69ae, 42895a9bcba64aa484f6509605035d02; no voters: 7962f587ccd842ad8a3f6ed9a5d7707e, 8d90d61da62549208fe117b4384a469b, b037d17b21f8422fa5e41b70eb4dcd08, ec6c8d818e744cdbb6b0d15b321010e0
I20260812 06:36:36.257669  8439 raft_consensus.cc:2393] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 42895a9bcba64aa484f6509605035d02 in current term 12: Already voted for candidate 8d90d61da62549208fe117b4384a469b in this term.
I20260812 06:36:36.257696  8645 raft_consensus.cc:3060] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [term 11 FOLLOWER]: Advancing to term 12
I20260812 06:36:36.257825  8649 raft_consensus.cc:2749] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260812 06:36:36.258740  8645 raft_consensus.cc:2749] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260812 06:36:36.261037  8439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "2df4308c852240859ada53d216ab69ae" candidate_term: 12 candidate_status { last_received { term: 11 index: 1438 } } ignore_live_leader: false dest_uuid: "222b20de5a1145c6b8a11df8374e8790" is_pre_election: true
I20260812 06:36:36.261219  8439 raft_consensus.cc:2393] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2df4308c852240859ada53d216ab69ae in current term 12: Already voted for candidate 8d90d61da62549208fe117b4384a469b in this term.
W20260812 06:36:36.360669  8639 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:36:36.360899  8639 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:36:36.360947  8639 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 06:36:36.364511  8639 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:36:36.364624  8639 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.177.3
I20260812 06:36:36.368314  8639 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-2/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.177.20:34335
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--consensus_rpc_timeout_ms=1000
--log_cache_size_limit_mb=10
--fs_data_dirs=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.177.3:40897
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--webserver_interface=127.4.177.3
--webserver_port=44527
--tserver_master_addrs=127.4.177.62:42183
--never_fsync=true
--heap_profile_path=/tmp/kudu.8639
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.177.3

Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 06:36:36.369331  8639 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:36:36.370527  8639 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 06:36:36.377565  8674 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:36:36.377563  8669 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:36:36.377559  8676 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:36:36.377820  8639 server_base.cc:1061] running on GCE node
I20260812 06:36:36.378336  8639 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:36:36.378931  8639 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:36:36.380138  8639 hybrid_clock.cc:648] HybridClock initialized: now 1786516596380118 us; error 33 us; skew 500 ppm
I20260812 06:36:36.382026  8639 webserver.cc:533] Webserver started at http://127.4.177.3:44527/ using document root <none> and password file <none>
I20260812 06:36:36.382676  8639 fs_manager.cc:362] Metadata directory not provided
I20260812 06:36:36.382777  8639 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:36:36.386345  8639 fs_manager.cc:714] Time spent opening directory manager: real 0.002s	user 0.004s	sys 0.000s
I20260812 06:36:36.388322  8692 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 06:36:36.389333  8639 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260812 06:36:36.389467  8639 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-2/data,/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-2/wal
uuid: "7962f587ccd842ad8a3f6ed9a5d7707e"
format_stamp: "Formatted at 2026-08-12 06:36:11 on dist-test-slave-jztv"
I20260812 06:36:36.389868  8639 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-2/wal
metadata directory: /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-2/wal
1 data directories: /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:36:36.420368  8639 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 06:36:36.421351  8639 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:36:36.421548  8639 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:36:36.422243  8639 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260812 06:36:36.424678  8702 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
I20260812 06:36:36.428798  8639 ts_tablet_manager.cc:585] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260812 06:36:36.428874  8639 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.005s	user 0.001s	sys 0.000s
I20260812 06:36:36.428958  8639 ts_tablet_manager.cc:600] Registering tablets (0/1 complete)
I20260812 06:36:36.430516  8639 ts_tablet_manager.cc:616] Registered 1 tablets
I20260812 06:36:36.430584  8639 ts_tablet_manager.cc:595] Time spent register tablets: real 0.002s	user 0.001s	sys 0.000s
I20260812 06:36:36.430904  8702 tablet_bootstrap.cc:492] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e: Bootstrap starting.
I20260812 06:36:36.441969  8877 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.4.177.3:40897 every 8 connection(s)
I20260812 06:36:36.441949  8639 rpc_server.cc:307] RPC server started. Bound to: 127.4.177.3:40897
I20260812 06:36:36.443331  8639 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-2/data/info.pb
I20260812 06:36:36.443504  4804 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskCnZu03/build/debug/bin/kudu as pid 8639
I20260812 06:36:36.454008  8878 heartbeater.cc:344] Connected to a master server at 127.4.177.62:42183
I20260812 06:36:36.454294  8878 heartbeater.cc:461] Registering TS with master...
I20260812 06:36:36.454800  8878 heartbeater.cc:507] Master 127.4.177.62:42183 requested a full tablet report, sending...
I20260812 06:36:36.456981  5812 ts_manager.cc:194] Re-registered known tserver with Master: 7962f587ccd842ad8a3f6ed9a5d7707e (127.4.177.3:40897)
I20260812 06:36:36.457954  5812 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.4.177.3:42999
I20260812 06:36:36.528581  8702 log.cc:826] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e: Log is configured to *not* fsync() on all Append() calls
W20260812 06:36:36.735306  6150 consensus_peers.cc:597] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b -> Peer 7962f587ccd842ad8a3f6ed9a5d7707e (127.4.177.3:40897): Couldn't send request to peer 7962f587ccd842ad8a3f6ed9a5d7707e. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260812 06:36:36.762423  6703 raft_consensus.cc:1275] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [term 12 FOLLOWER]: Refusing update from remote peer 8d90d61da62549208fe117b4384a469b: Log matching property violated. Preceding OpId in replica: term: 11 index: 1438. Preceding OpId from leader: term: 12 index: 1431. (term mismatch)
I20260812 06:36:36.762580  6703 pending_rounds.cc:85] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02: Aborting all ops after (but not including) 1430
I20260812 06:36:36.762621  6703 pending_rounds.cc:107] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02: Aborting uncommitted WRITE_OP operation due to leader change: 11.1431
I20260812 06:36:36.762881  6703 pending_rounds.cc:107] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02: Aborting uncommitted WRITE_OP operation due to leader change: 11.1432
I20260812 06:36:36.763068  6703 pending_rounds.cc:107] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02: Aborting uncommitted WRITE_OP operation due to leader change: 11.1433
I20260812 06:36:36.763226  6703 pending_rounds.cc:107] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02: Aborting uncommitted WRITE_OP operation due to leader change: 11.1434
I20260812 06:36:36.763242  7321 raft_consensus.cc:1275] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08 [term 12 FOLLOWER]: Refusing update from remote peer 8d90d61da62549208fe117b4384a469b: Log matching property violated. Preceding OpId in replica: term: 11 index: 1391. Preceding OpId from leader: term: 12 index: 1431. (index mismatch)
I20260812 06:36:36.763378  6703 pending_rounds.cc:107] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02: Aborting uncommitted WRITE_OP operation due to leader change: 11.1435
I20260812 06:36:36.763561  6703 pending_rounds.cc:107] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02: Aborting uncommitted WRITE_OP operation due to leader change: 11.1436
I20260812 06:36:36.763746  6703 pending_rounds.cc:107] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02: Aborting uncommitted WRITE_OP operation due to leader change: 11.1437
I20260812 06:36:36.763743  8631 consensus_queue.cc:1048] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" member_type: VOTER last_known_addr { host: "127.4.177.7" port: 46299 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1431, Last known committed idx: 1386, Time since last communication: 0.000s
I20260812 06:36:36.763919  6703 pending_rounds.cc:107] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02: Aborting uncommitted WRITE_OP operation due to leader change: 11.1438
I20260812 06:36:36.764847  8631 consensus_queue.cc:1048] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b [LEADER]: Connected to new peer: Peer: permanent_uuid: "42895a9bcba64aa484f6509605035d02" member_type: VOTER last_known_addr { host: "127.4.177.4" port: 38181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1431, Last known committed idx: 1430, Time since last communication: 0.000s
I20260812 06:36:36.822731  8439 raft_consensus.cc:1275] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [term 12 FOLLOWER]: Refusing update from remote peer 8d90d61da62549208fe117b4384a469b: Log matching property violated. Preceding OpId in replica: term: 10 index: 1307. Preceding OpId from leader: term: 12 index: 1431. (index mismatch)
I20260812 06:36:36.823458  8885 consensus_queue.cc:1048] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b [LEADER]: Connected to new peer: Peer: permanent_uuid: "222b20de5a1145c6b8a11df8374e8790" member_type: VOTER last_known_addr { host: "127.4.177.5" port: 43521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1431, Last known committed idx: 1299, Time since last communication: 0.000s
I20260812 06:36:36.823613  8885 consensus_queue.cc:1243] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b [LEADER]: Peer 222b20de5a1145c6b8a11df8374e8790 log is divergent from this leader: its last log entry 10.1307 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1299
I20260812 06:36:36.827070  8439 raft_consensus.cc:1217] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [term 12 FOLLOWER]: Deduplicated request from leader. Original: 10.1299->[10.1300-12.1431]   Dedup: 10.1302->[11.1303-12.1431]
I20260812 06:36:36.827251  8439 pending_rounds.cc:85] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790: Aborting all ops after (but not including) 1302
I20260812 06:36:36.827342  8439 pending_rounds.cc:107] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790: Aborting uncommitted WRITE_OP operation due to leader change: 10.1303
I20260812 06:36:36.827837  8439 pending_rounds.cc:107] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790: Aborting uncommitted WRITE_OP operation due to leader change: 10.1304
I20260812 06:36:36.828047  8439 pending_rounds.cc:107] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790: Aborting uncommitted WRITE_OP operation due to leader change: 10.1305
I20260812 06:36:36.828234  8439 pending_rounds.cc:107] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790: Aborting uncommitted WRITE_OP operation due to leader change: 10.1306
I20260812 06:36:36.828418  8439 pending_rounds.cc:107] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790: Aborting uncommitted WRITE_OP operation due to leader change: 10.1307
I20260812 06:36:36.832036  6046 raft_consensus.cc:1275] T 7065bb403e3047dd9a0853c49318ef1f P ec6c8d818e744cdbb6b0d15b321010e0 [term 12 FOLLOWER]: Refusing update from remote peer 8d90d61da62549208fe117b4384a469b: Log matching property violated. Preceding OpId in replica: term: 11 index: 1366. Preceding OpId from leader: term: 12 index: 1431. (index mismatch)
I20260812 06:36:36.832551  8885 consensus_queue.cc:1048] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 41057 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1431, Last known committed idx: 1358, Time since last communication: 0.000s
I20260812 06:36:36.856505  7099 raft_consensus.cc:1275] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [term 12 FOLLOWER]: Refusing update from remote peer 8d90d61da62549208fe117b4384a469b: Log matching property violated. Preceding OpId in replica: term: 11 index: 1438. Preceding OpId from leader: term: 12 index: 1431. (term mismatch)
I20260812 06:36:36.856626  7099 pending_rounds.cc:85] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae: Aborting all ops after (but not including) 1430
I20260812 06:36:36.856657  7099 pending_rounds.cc:107] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae: Aborting uncommitted WRITE_OP operation due to leader change: 11.1431
I20260812 06:36:36.856856  7099 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:50492 (ReqId={client: 3067f3aef2a44cc4bb7f2ccd6acf83db, seq_no=1423, attempt_no=1}) took 1651 ms. Trace:
I20260812 06:36:36.857447  7099 rpcz_store.cc:276] 0812 06:36:35.205263 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 06:36:35.205323 (+    60us) service_pool.cc:224] Handling call
0812 06:36:36.856843 (+1651520us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260812 06:36:36.857723  7099 pending_rounds.cc:107] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae: Aborting uncommitted WRITE_OP operation due to leader change: 11.1432
I20260812 06:36:36.857915  7099 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:50492 (ReqId={client: 3067f3aef2a44cc4bb7f2ccd6acf83db, seq_no=1424, attempt_no=1}) took 1649 ms. Trace:
I20260812 06:36:36.857981  7099 rpcz_store.cc:276] 0812 06:36:35.208538 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 06:36:35.208771 (+   233us) service_pool.cc:224] Handling call
0812 06:36:36.857903 (+1649132us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260812 06:36:36.858145  7099 pending_rounds.cc:107] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae: Aborting uncommitted WRITE_OP operation due to leader change: 11.1433
I20260812 06:36:36.858325  7099 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:50492 (ReqId={client: 3067f3aef2a44cc4bb7f2ccd6acf83db, seq_no=1425, attempt_no=1}) took 1653 ms. Trace:
I20260812 06:36:36.858398  7099 rpcz_store.cc:276] 0812 06:36:35.205156 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 06:36:35.205217 (+    61us) service_pool.cc:224] Handling call
0812 06:36:36.858316 (+1653099us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260812 06:36:36.858544  7099 pending_rounds.cc:107] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae: Aborting uncommitted WRITE_OP operation due to leader change: 11.1434
I20260812 06:36:36.858712  7099 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:50492 (ReqId={client: 3067f3aef2a44cc4bb7f2ccd6acf83db, seq_no=1426, attempt_no=1}) took 1650 ms. Trace:
I20260812 06:36:36.858773  7099 rpcz_store.cc:276] 0812 06:36:35.208617 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 06:36:35.208665 (+    48us) service_pool.cc:224] Handling call
0812 06:36:36.858705 (+1650040us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260812 06:36:36.858913  7099 pending_rounds.cc:107] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae: Aborting uncommitted WRITE_OP operation due to leader change: 11.1435
I20260812 06:36:36.859064  7099 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:50492 (ReqId={client: 3067f3aef2a44cc4bb7f2ccd6acf83db, seq_no=1427, attempt_no=1}) took 1650 ms. Trace:
I20260812 06:36:36.859131  7099 rpcz_store.cc:276] 0812 06:36:35.208759 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 06:36:35.208815 (+    56us) service_pool.cc:224] Handling call
0812 06:36:36.859057 (+1650242us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260812 06:36:36.859283  7099 pending_rounds.cc:107] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae: Aborting uncommitted WRITE_OP operation due to leader change: 11.1436
I20260812 06:36:36.859475  7099 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:50492 (ReqId={client: 3067f3aef2a44cc4bb7f2ccd6acf83db, seq_no=1428, attempt_no=1}) took 1652 ms. Trace:
I20260812 06:36:36.859539  7099 rpcz_store.cc:276] 0812 06:36:35.206611 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 06:36:35.206754 (+   143us) service_pool.cc:224] Handling call
0812 06:36:36.859468 (+1652714us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260812 06:36:36.859697  7099 pending_rounds.cc:107] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae: Aborting uncommitted WRITE_OP operation due to leader change: 11.1437
I20260812 06:36:36.859853  7099 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:50492 (ReqId={client: 3067f3aef2a44cc4bb7f2ccd6acf83db, seq_no=1429, attempt_no=1}) took 1653 ms. Trace:
I20260812 06:36:36.859913  7099 rpcz_store.cc:276] 0812 06:36:35.206371 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 06:36:35.206477 (+   106us) service_pool.cc:224] Handling call
0812 06:36:36.859845 (+1653368us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260812 06:36:36.860072  7099 pending_rounds.cc:107] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae: Aborting uncommitted WRITE_OP operation due to leader change: 11.1438
I20260812 06:36:36.860217  7099 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:50492 (ReqId={client: 3067f3aef2a44cc4bb7f2ccd6acf83db, seq_no=1430, attempt_no=1}) took 1651 ms. Trace:
I20260812 06:36:36.860272  7099 rpcz_store.cc:276] 0812 06:36:35.208701 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 06:36:35.208757 (+    56us) service_pool.cc:224] Handling call
0812 06:36:36.860209 (+1651452us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260812 06:36:36.863116  8631 consensus_queue.cc:1048] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b [LEADER]: Connected to new peer: Peer: permanent_uuid: "2df4308c852240859ada53d216ab69ae" member_type: VOTER last_known_addr { host: "127.4.177.6" port: 34269 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1431, Last known committed idx: 1430, Time since last communication: 0.000s
W20260812 06:36:36.865825  7063 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50492: Illegal state: replica 2df4308c852240859ada53d216ab69ae is not leader of this config: current role FOLLOWER
W20260812 06:36:36.866421  7061 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50492: Illegal state: replica 2df4308c852240859ada53d216ab69ae is not leader of this config: current role FOLLOWER
W20260812 06:36:36.868273  7060 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50492: Illegal state: replica 2df4308c852240859ada53d216ab69ae is not leader of this config: current role FOLLOWER
W20260812 06:36:36.869670  7061 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50492: Illegal state: replica 2df4308c852240859ada53d216ab69ae is not leader of this config: current role FOLLOWER
W20260812 06:36:36.870726  7059 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50492: Illegal state: replica 2df4308c852240859ada53d216ab69ae is not leader of this config: current role FOLLOWER
W20260812 06:36:36.871291  7060 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50492: Illegal state: replica 2df4308c852240859ada53d216ab69ae is not leader of this config: current role FOLLOWER
W20260812 06:36:36.871810  7063 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50492: Illegal state: replica 2df4308c852240859ada53d216ab69ae is not leader of this config: current role FOLLOWER
W20260812 06:36:36.873114  6626 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:42408: Illegal state: replica 42895a9bcba64aa484f6509605035d02 is not leader of this config: current role FOLLOWER
W20260812 06:36:36.874191  7061 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50492: Illegal state: replica 2df4308c852240859ada53d216ab69ae is not leader of this config: current role FOLLOWER
W20260812 06:36:36.875588  6626 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:42408: Illegal state: replica 42895a9bcba64aa484f6509605035d02 is not leader of this config: current role FOLLOWER
W20260812 06:36:36.879405  6625 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:42408: Illegal state: replica 42895a9bcba64aa484f6509605035d02 is not leader of this config: current role FOLLOWER
W20260812 06:36:36.881292  6625 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:42408: Illegal state: replica 42895a9bcba64aa484f6509605035d02 is not leader of this config: current role FOLLOWER
W20260812 06:36:36.882328  6621 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:42408: Illegal state: replica 42895a9bcba64aa484f6509605035d02 is not leader of this config: current role FOLLOWER
W20260812 06:36:36.886862  6626 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:42408: Illegal state: replica 42895a9bcba64aa484f6509605035d02 is not leader of this config: current role FOLLOWER
I20260812 06:36:36.958077  8911 mvcc.cc:204] Tried to move back new op lower bound from 7317571971993108480 to 7317571967727996928. Current Snapshot: MvccSnapshot[applied={T|T < 7317571971993108480}]
I20260812 06:36:37.464175  8878 heartbeater.cc:499] Master 127.4.177.62:42183 was elected leader, sending a full tablet report...
I20260812 06:36:38.305768  8702 tablet_bootstrap.cc:492] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e: Bootstrap replayed 1/1 log segments. Stats: ops{read=1449 overwritten=11 applied=1430 ignored=0} inserts{seen=14230 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 8 replicates
I20260812 06:36:38.306452  8702 tablet_bootstrap.cc:492] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e: Bootstrap complete.
I20260812 06:36:38.307659  8702 ts_tablet_manager.cc:1403] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e: Time spent bootstrapping tablet: real 1.877s	user 1.248s	sys 0.071s
I20260812 06:36:38.309640  8702 raft_consensus.cc:359] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [term 11 FOLLOWER]: Replica starting. Triggering 8 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df4308c852240859ada53d216ab69ae" member_type: VOTER last_known_addr { host: "127.4.177.6" port: 34269 } } peers { permanent_uuid: "42895a9bcba64aa484f6509605035d02" member_type: VOTER last_known_addr { host: "127.4.177.4" port: 38181 } } peers { permanent_uuid: "222b20de5a1145c6b8a11df8374e8790" member_type: VOTER last_known_addr { host: "127.4.177.5" port: 43521 } } peers { permanent_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 40897 } } peers { permanent_uuid: "8d90d61da62549208fe117b4384a469b" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 41539 } } peers { permanent_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" member_type: VOTER last_known_addr { host: "127.4.177.7" port: 46299 } } peers { permanent_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 41057 } }
I20260812 06:36:38.311798  8702 raft_consensus.cc:740] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [term 11 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7962f587ccd842ad8a3f6ed9a5d7707e, State: Initialized, Role: FOLLOWER
I20260812 06:36:38.312299  8702 consensus_queue.cc:260] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1430, Last appended: 11.1438, Last appended by leader: 1438, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df4308c852240859ada53d216ab69ae" member_type: VOTER last_known_addr { host: "127.4.177.6" port: 34269 } } peers { permanent_uuid: "42895a9bcba64aa484f6509605035d02" member_type: VOTER last_known_addr { host: "127.4.177.4" port: 38181 } } peers { permanent_uuid: "222b20de5a1145c6b8a11df8374e8790" member_type: VOTER last_known_addr { host: "127.4.177.5" port: 43521 } } peers { permanent_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 40897 } } peers { permanent_uuid: "8d90d61da62549208fe117b4384a469b" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 41539 } } peers { permanent_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" member_type: VOTER last_known_addr { host: "127.4.177.7" port: 46299 } } peers { permanent_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 41057 } }
I20260812 06:36:38.313114  8702 ts_tablet_manager.cc:1434] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e: Time spent starting tablet: real 0.005s	user 0.006s	sys 0.000s
I20260812 06:36:39.107839  8976 raft_consensus.cc:493] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 8d90d61da62549208fe117b4384a469b)
I20260812 06:36:39.107999  8976 raft_consensus.cc:515] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df4308c852240859ada53d216ab69ae" member_type: VOTER last_known_addr { host: "127.4.177.6" port: 34269 } } peers { permanent_uuid: "42895a9bcba64aa484f6509605035d02" member_type: VOTER last_known_addr { host: "127.4.177.4" port: 38181 } } peers { permanent_uuid: "222b20de5a1145c6b8a11df8374e8790" member_type: VOTER last_known_addr { host: "127.4.177.5" port: 43521 } } peers { permanent_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 40897 } } peers { permanent_uuid: "8d90d61da62549208fe117b4384a469b" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 41539 } } peers { permanent_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" member_type: VOTER last_known_addr { host: "127.4.177.7" port: 46299 } } peers { permanent_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 41057 } }
I20260812 06:36:39.108795  8976 leader_election.cc:290] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 2df4308c852240859ada53d216ab69ae (127.4.177.6:34269), 42895a9bcba64aa484f6509605035d02 (127.4.177.4:38181), 222b20de5a1145c6b8a11df8374e8790 (127.4.177.5:43521), 7962f587ccd842ad8a3f6ed9a5d7707e (127.4.177.3:40897), 8d90d61da62549208fe117b4384a469b (127.4.177.2:41539), ec6c8d818e744cdbb6b0d15b321010e0 (127.4.177.1:41057)
I20260812 06:36:39.109161  7099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" candidate_term: 13 candidate_status { last_received { term: 12 index: 1581 } } ignore_live_leader: false dest_uuid: "2df4308c852240859ada53d216ab69ae" is_pre_election: true
I20260812 06:36:39.109160  6701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" candidate_term: 13 candidate_status { last_received { term: 12 index: 1581 } } ignore_live_leader: false dest_uuid: "42895a9bcba64aa484f6509605035d02" is_pre_election: true
I20260812 06:36:39.109359  8439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" candidate_term: 13 candidate_status { last_received { term: 12 index: 1581 } } ignore_live_leader: false dest_uuid: "222b20de5a1145c6b8a11df8374e8790" is_pre_election: true
I20260812 06:36:39.110023  6046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" candidate_term: 13 candidate_status { last_received { term: 12 index: 1581 } } ignore_live_leader: false dest_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" is_pre_election: true
I20260812 06:36:39.110536  7211 leader_election.cc:304] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 5 responses out of 7 voters: 1 yes votes; 4 no votes. yes voters: b037d17b21f8422fa5e41b70eb4dcd08; no voters: 222b20de5a1145c6b8a11df8374e8790, 2df4308c852240859ada53d216ab69ae, 42895a9bcba64aa484f6509605035d02, ec6c8d818e744cdbb6b0d15b321010e0
I20260812 06:36:39.110761  8976 raft_consensus.cc:2749] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260812 06:36:39.113276  8807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" candidate_term: 13 candidate_status { last_received { term: 12 index: 1581 } } ignore_live_leader: false dest_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" is_pre_election: true
I20260812 06:36:39.113588  8807 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b037d17b21f8422fa5e41b70eb4dcd08 in term 11.
I20260812 06:36:39.337944  8980 raft_consensus.cc:493] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 8d90d61da62549208fe117b4384a469b)
I20260812 06:36:39.338110  8980 raft_consensus.cc:515] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df4308c852240859ada53d216ab69ae" member_type: VOTER last_known_addr { host: "127.4.177.6" port: 34269 } } peers { permanent_uuid: "42895a9bcba64aa484f6509605035d02" member_type: VOTER last_known_addr { host: "127.4.177.4" port: 38181 } } peers { permanent_uuid: "222b20de5a1145c6b8a11df8374e8790" member_type: VOTER last_known_addr { host: "127.4.177.5" port: 43521 } } peers { permanent_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 40897 } } peers { permanent_uuid: "8d90d61da62549208fe117b4384a469b" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 41539 } } peers { permanent_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" member_type: VOTER last_known_addr { host: "127.4.177.7" port: 46299 } } peers { permanent_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 41057 } }
I20260812 06:36:39.339656  8980 leader_election.cc:290] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 2df4308c852240859ada53d216ab69ae (127.4.177.6:34269), 42895a9bcba64aa484f6509605035d02 (127.4.177.4:38181), 7962f587ccd842ad8a3f6ed9a5d7707e (127.4.177.3:40897), 8d90d61da62549208fe117b4384a469b (127.4.177.2:41539), b037d17b21f8422fa5e41b70eb4dcd08 (127.4.177.7:46299), ec6c8d818e744cdbb6b0d15b321010e0 (127.4.177.1:41057)
I20260812 06:36:39.345949  7099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "222b20de5a1145c6b8a11df8374e8790" candidate_term: 13 candidate_status { last_received { term: 12 index: 1656 } } ignore_live_leader: false dest_uuid: "2df4308c852240859ada53d216ab69ae" is_pre_election: true
I20260812 06:36:39.346097  8807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "222b20de5a1145c6b8a11df8374e8790" candidate_term: 13 candidate_status { last_received { term: 12 index: 1656 } } ignore_live_leader: false dest_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" is_pre_election: true
I20260812 06:36:39.346284  8807 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 222b20de5a1145c6b8a11df8374e8790 in term 11.
I20260812 06:36:39.346575  7322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "222b20de5a1145c6b8a11df8374e8790" candidate_term: 13 candidate_status { last_received { term: 12 index: 1656 } } ignore_live_leader: false dest_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" is_pre_election: true
I20260812 06:36:39.346730  7322 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 222b20de5a1145c6b8a11df8374e8790 in term 12.
I20260812 06:36:39.347157  6046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "222b20de5a1145c6b8a11df8374e8790" candidate_term: 13 candidate_status { last_received { term: 12 index: 1656 } } ignore_live_leader: false dest_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" is_pre_election: true
I20260812 06:36:39.347546  6701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "222b20de5a1145c6b8a11df8374e8790" candidate_term: 13 candidate_status { last_received { term: 12 index: 1656 } } ignore_live_leader: false dest_uuid: "42895a9bcba64aa484f6509605035d02" is_pre_election: true
I20260812 06:36:39.348202  8997 raft_consensus.cc:493] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 8d90d61da62549208fe117b4384a469b)
I20260812 06:36:39.348297  8997 raft_consensus.cc:515] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df4308c852240859ada53d216ab69ae" member_type: VOTER last_known_addr { host: "127.4.177.6" port: 34269 } } peers { permanent_uuid: "42895a9bcba64aa484f6509605035d02" member_type: VOTER last_known_addr { host: "127.4.177.4" port: 38181 } } peers { permanent_uuid: "222b20de5a1145c6b8a11df8374e8790" member_type: VOTER last_known_addr { host: "127.4.177.5" port: 43521 } } peers { permanent_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 40897 } } peers { permanent_uuid: "8d90d61da62549208fe117b4384a469b" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 41539 } } peers { permanent_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" member_type: VOTER last_known_addr { host: "127.4.177.7" port: 46299 } } peers { permanent_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 41057 } }
I20260812 06:36:39.348950  8997 leader_election.cc:290] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 42895a9bcba64aa484f6509605035d02 (127.4.177.4:38181), 222b20de5a1145c6b8a11df8374e8790 (127.4.177.5:43521), 7962f587ccd842ad8a3f6ed9a5d7707e (127.4.177.3:40897), 8d90d61da62549208fe117b4384a469b (127.4.177.2:41539), b037d17b21f8422fa5e41b70eb4dcd08 (127.4.177.7:46299), ec6c8d818e744cdbb6b0d15b321010e0 (127.4.177.1:41057)
I20260812 06:36:39.349262  9002 raft_consensus.cc:493] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 8d90d61da62549208fe117b4384a469b)
I20260812 06:36:39.349270  7322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "2df4308c852240859ada53d216ab69ae" candidate_term: 13 candidate_status { last_received { term: 12 index: 1661 } } ignore_live_leader: false dest_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" is_pre_election: true
I20260812 06:36:39.349407  7322 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2df4308c852240859ada53d216ab69ae in term 12.
I20260812 06:36:39.349403  9002 raft_consensus.cc:515] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df4308c852240859ada53d216ab69ae" member_type: VOTER last_known_addr { host: "127.4.177.6" port: 34269 } } peers { permanent_uuid: "42895a9bcba64aa484f6509605035d02" member_type: VOTER last_known_addr { host: "127.4.177.4" port: 38181 } } peers { permanent_uuid: "222b20de5a1145c6b8a11df8374e8790" member_type: VOTER last_known_addr { host: "127.4.177.5" port: 43521 } } peers { permanent_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 40897 } } peers { permanent_uuid: "8d90d61da62549208fe117b4384a469b" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 41539 } } peers { permanent_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" member_type: VOTER last_known_addr { host: "127.4.177.7" port: 46299 } } peers { permanent_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 41057 } }
I20260812 06:36:39.349603  8439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "2df4308c852240859ada53d216ab69ae" candidate_term: 13 candidate_status { last_received { term: 12 index: 1661 } } ignore_live_leader: false dest_uuid: "222b20de5a1145c6b8a11df8374e8790" is_pre_election: true
I20260812 06:36:39.349726  8439 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2df4308c852240859ada53d216ab69ae in term 12.
I20260812 06:36:39.349932  6701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "2df4308c852240859ada53d216ab69ae" candidate_term: 13 candidate_status { last_received { term: 12 index: 1661 } } ignore_live_leader: false dest_uuid: "42895a9bcba64aa484f6509605035d02" is_pre_election: true
I20260812 06:36:39.350039  6701 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2df4308c852240859ada53d216ab69ae in term 12.
I20260812 06:36:39.350353  7020 leader_election.cc:304] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 7 voters: 4 yes votes; 0 no votes. yes voters: 222b20de5a1145c6b8a11df8374e8790, 2df4308c852240859ada53d216ab69ae, 42895a9bcba64aa484f6509605035d02, b037d17b21f8422fa5e41b70eb4dcd08; no voters: 
I20260812 06:36:39.350342  6046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "2df4308c852240859ada53d216ab69ae" candidate_term: 13 candidate_status { last_received { term: 12 index: 1661 } } ignore_live_leader: false dest_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" is_pre_election: true
I20260812 06:36:39.350509  8997 raft_consensus.cc:2804] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260812 06:36:39.350574  8997 raft_consensus.cc:493] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [term 12 FOLLOWER]: Starting leader election (detected failure of leader 8d90d61da62549208fe117b4384a469b)
I20260812 06:36:39.350620  8997 raft_consensus.cc:3060] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [term 12 FOLLOWER]: Advancing to term 13
I20260812 06:36:39.350659  7099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "42895a9bcba64aa484f6509605035d02" candidate_term: 13 candidate_status { last_received { term: 12 index: 1661 } } ignore_live_leader: false dest_uuid: "2df4308c852240859ada53d216ab69ae" is_pre_election: true
I20260812 06:36:39.350917  8439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "42895a9bcba64aa484f6509605035d02" candidate_term: 13 candidate_status { last_received { term: 12 index: 1661 } } ignore_live_leader: false dest_uuid: "222b20de5a1145c6b8a11df8374e8790" is_pre_election: true
I20260812 06:36:39.351016  8439 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 42895a9bcba64aa484f6509605035d02 in term 12.
I20260812 06:36:39.351078  9002 leader_election.cc:290] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 2df4308c852240859ada53d216ab69ae (127.4.177.6:34269), 222b20de5a1145c6b8a11df8374e8790 (127.4.177.5:43521), 7962f587ccd842ad8a3f6ed9a5d7707e (127.4.177.3:40897), 8d90d61da62549208fe117b4384a469b (127.4.177.2:41539), b037d17b21f8422fa5e41b70eb4dcd08 (127.4.177.7:46299), ec6c8d818e744cdbb6b0d15b321010e0 (127.4.177.1:41057)
I20260812 06:36:39.351408  7322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "42895a9bcba64aa484f6509605035d02" candidate_term: 13 candidate_status { last_received { term: 12 index: 1661 } } ignore_live_leader: false dest_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" is_pre_election: true
I20260812 06:36:39.351495  6046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "42895a9bcba64aa484f6509605035d02" candidate_term: 13 candidate_status { last_received { term: 12 index: 1661 } } ignore_live_leader: false dest_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" is_pre_election: true
I20260812 06:36:39.351519  7322 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 42895a9bcba64aa484f6509605035d02 in term 12.
I20260812 06:36:39.352232  8997 raft_consensus.cc:515] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df4308c852240859ada53d216ab69ae" member_type: VOTER last_known_addr { host: "127.4.177.6" port: 34269 } } peers { permanent_uuid: "42895a9bcba64aa484f6509605035d02" member_type: VOTER last_known_addr { host: "127.4.177.4" port: 38181 } } peers { permanent_uuid: "222b20de5a1145c6b8a11df8374e8790" member_type: VOTER last_known_addr { host: "127.4.177.5" port: 43521 } } peers { permanent_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 40897 } } peers { permanent_uuid: "8d90d61da62549208fe117b4384a469b" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 41539 } } peers { permanent_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" member_type: VOTER last_known_addr { host: "127.4.177.7" port: 46299 } } peers { permanent_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 41057 } }
I20260812 06:36:39.352558  7099 raft_consensus.cc:2393] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 42895a9bcba64aa484f6509605035d02 in current term 13: Already voted for candidate 2df4308c852240859ada53d216ab69ae in this term.
I20260812 06:36:39.353276  7322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "2df4308c852240859ada53d216ab69ae" candidate_term: 13 candidate_status { last_received { term: 12 index: 1661 } } ignore_live_leader: false dest_uuid: "b037d17b21f8422fa5e41b70eb4dcd08"
I20260812 06:36:39.353380  7322 raft_consensus.cc:3060] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08 [term 12 FOLLOWER]: Advancing to term 13
I20260812 06:36:39.353437  8439 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "2df4308c852240859ada53d216ab69ae" candidate_term: 13 candidate_status { last_received { term: 12 index: 1661 } } ignore_live_leader: false dest_uuid: "222b20de5a1145c6b8a11df8374e8790"
I20260812 06:36:39.353530  8439 raft_consensus.cc:3060] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [term 12 FOLLOWER]: Advancing to term 13
I20260812 06:36:39.353999  6701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "2df4308c852240859ada53d216ab69ae" candidate_term: 13 candidate_status { last_received { term: 12 index: 1661 } } ignore_live_leader: false dest_uuid: "42895a9bcba64aa484f6509605035d02"
I20260812 06:36:39.354097  6701 raft_consensus.cc:3060] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [term 12 FOLLOWER]: Advancing to term 13
I20260812 06:36:39.354571  7322 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2df4308c852240859ada53d216ab69ae in term 13.
I20260812 06:36:39.354571  8439 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2df4308c852240859ada53d216ab69ae in term 13.
I20260812 06:36:39.354985  8997 leader_election.cc:290] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [CANDIDATE]: Term 13 election: Requested vote from peers 42895a9bcba64aa484f6509605035d02 (127.4.177.4:38181), 222b20de5a1145c6b8a11df8374e8790 (127.4.177.5:43521), 7962f587ccd842ad8a3f6ed9a5d7707e (127.4.177.3:40897), 8d90d61da62549208fe117b4384a469b (127.4.177.2:41539), b037d17b21f8422fa5e41b70eb4dcd08 (127.4.177.7:46299), ec6c8d818e744cdbb6b0d15b321010e0 (127.4.177.1:41057)
I20260812 06:36:39.355134  6701 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2df4308c852240859ada53d216ab69ae in term 13.
I20260812 06:36:39.355239  8807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "2df4308c852240859ada53d216ab69ae" candidate_term: 13 candidate_status { last_received { term: 12 index: 1661 } } ignore_live_leader: false dest_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" is_pre_election: true
I20260812 06:36:39.355383  8807 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2df4308c852240859ada53d216ab69ae in term 11.
I20260812 06:36:39.355401  6046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "2df4308c852240859ada53d216ab69ae" candidate_term: 13 candidate_status { last_received { term: 12 index: 1661 } } ignore_live_leader: false dest_uuid: "ec6c8d818e744cdbb6b0d15b321010e0"
I20260812 06:36:39.355482  7020 leader_election.cc:304] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 7 voters: 4 yes votes; 0 no votes. yes voters: 222b20de5a1145c6b8a11df8374e8790, 2df4308c852240859ada53d216ab69ae, 42895a9bcba64aa484f6509605035d02, b037d17b21f8422fa5e41b70eb4dcd08; no voters: 
I20260812 06:36:39.355623  8805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "2df4308c852240859ada53d216ab69ae" candidate_term: 13 candidate_status { last_received { term: 12 index: 1661 } } ignore_live_leader: false dest_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e"
I20260812 06:36:39.355726  8805 raft_consensus.cc:3060] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [term 11 FOLLOWER]: Advancing to term 13
I20260812 06:36:39.357623  8805 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2df4308c852240859ada53d216ab69ae in term 13.
I20260812 06:36:39.358011  8997 raft_consensus.cc:2804] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [term 13 FOLLOWER]: Leader election won for term 13
I20260812 06:36:39.358206  8997 raft_consensus.cc:697] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [term 13 LEADER]: Becoming Leader. State: Replica: 2df4308c852240859ada53d216ab69ae, State: Running, Role: LEADER
I20260812 06:36:39.358494  8805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "42895a9bcba64aa484f6509605035d02" candidate_term: 13 candidate_status { last_received { term: 12 index: 1661 } } ignore_live_leader: false dest_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" is_pre_election: true
I20260812 06:36:39.358664  8805 raft_consensus.cc:2393] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 42895a9bcba64aa484f6509605035d02 in current term 13: Already voted for candidate 2df4308c852240859ada53d216ab69ae in this term.
I20260812 06:36:39.358628  8997 consensus_queue.cc:237] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1653, Committed index: 1653, Last appended: 12.1661, Last appended by leader: 1661, Current term: 13, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df4308c852240859ada53d216ab69ae" member_type: VOTER last_known_addr { host: "127.4.177.6" port: 34269 } } peers { permanent_uuid: "42895a9bcba64aa484f6509605035d02" member_type: VOTER last_known_addr { host: "127.4.177.4" port: 38181 } } peers { permanent_uuid: "222b20de5a1145c6b8a11df8374e8790" member_type: VOTER last_known_addr { host: "127.4.177.5" port: 43521 } } peers { permanent_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 40897 } } peers { permanent_uuid: "8d90d61da62549208fe117b4384a469b" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 41539 } } peers { permanent_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" member_type: VOTER last_known_addr { host: "127.4.177.7" port: 46299 } } peers { permanent_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 41057 } }
I20260812 06:36:39.361006  5808 catalog_manager.cc:5719] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae reported cstate change: term changed from 12 to 13, leader changed from 8d90d61da62549208fe117b4384a469b (127.4.177.2) to 2df4308c852240859ada53d216ab69ae (127.4.177.6). New cstate: current_term: 13 leader_uuid: "2df4308c852240859ada53d216ab69ae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df4308c852240859ada53d216ab69ae" member_type: VOTER last_known_addr { host: "127.4.177.6" port: 34269 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "42895a9bcba64aa484f6509605035d02" member_type: VOTER last_known_addr { host: "127.4.177.4" port: 38181 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "222b20de5a1145c6b8a11df8374e8790" member_type: VOTER last_known_addr { host: "127.4.177.5" port: 43521 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 40897 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8d90d61da62549208fe117b4384a469b" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 41539 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" member_type: VOTER last_known_addr { host: "127.4.177.7" port: 46299 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 41057 } health_report { overall_health: UNKNOWN } } }
I20260812 06:36:39.420476  6249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "2df4308c852240859ada53d216ab69ae" candidate_term: 13 candidate_status { last_received { term: 12 index: 1661 } } ignore_live_leader: false dest_uuid: "8d90d61da62549208fe117b4384a469b" is_pre_election: true
W20260812 06:36:39.420820  6150 connection.cc:256] RPC call timeout handler was delayed by -1.0137225431390107s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1s for a response
I20260812 06:36:39.421208  6249 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "42895a9bcba64aa484f6509605035d02" candidate_term: 13 candidate_status { last_received { term: 12 index: 1661 } } ignore_live_leader: false dest_uuid: "8d90d61da62549208fe117b4384a469b" is_pre_election: true
I20260812 06:36:39.422317  6248 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" candidate_term: 13 candidate_status { last_received { term: 12 index: 1581 } } ignore_live_leader: false dest_uuid: "8d90d61da62549208fe117b4384a469b" is_pre_election: true
I20260812 06:36:39.422785  6247 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "2df4308c852240859ada53d216ab69ae" candidate_term: 13 candidate_status { last_received { term: 12 index: 1661 } } ignore_live_leader: false dest_uuid: "8d90d61da62549208fe117b4384a469b"
I20260812 06:36:39.424463  8932 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:42054 (ReqId={client: 3067f3aef2a44cc4bb7f2ccd6acf83db, seq_no=1645, attempt_no=0}) took 1595 ms. Trace:
I20260812 06:36:39.424533  8932 rpcz_store.cc:276] 0812 06:36:37.828861 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 06:36:37.828975 (+   114us) service_pool.cc:224] Handling call
0812 06:36:39.424451 (+1595476us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0812 06:36:37.831002 (+     0us) write_op.cc:183] PREPARE: starting on tablet 7065bb403e3047dd9a0853c49318ef1f
0812 06:36:37.831089 (+    87us) write_op.cc:432] Acquiring schema lock in shared mode
0812 06:36:37.831095 (+     6us) write_op.cc:435] Acquired schema lock
0812 06:36:37.831096 (+     1us) tablet.cc:662] Decoding operations
0812 06:36:37.831254 (+   158us) write_op.cc:620] Acquiring the partition lock for write op
0812 06:36:37.831260 (+     6us) write_op.cc:641] Partition lock acquired for write op
0812 06:36:37.831263 (+     3us) tablet.cc:685] Acquiring locks for 10 operations
0812 06:36:37.831335 (+    72us) tablet.cc:701] Row locks acquired
0812 06:36:37.831337 (+     2us) write_op.cc:260] PREPARE: finished
0812 06:36:37.831365 (+    28us) write_op.cc:270] Start()
0812 06:36:37.831383 (+    18us) write_op.cc:276] Timestamp: P: 1786516597831343 usec, L: 0
0812 06:36:37.831385 (+     2us) op_driver.cc:348] REPLICATION: starting
0812 06:36:37.831506 (+   121us) log.cc:844] Serialized 686 byte log entry
0812 06:36:39.423844 (+1592338us) op_driver.cc:464] REPLICATION: finished
0812 06:36:39.423943 (+    99us) write_op.cc:301] APPLY: starting
0812 06:36:39.423962 (+    19us) tablet.cc:1367] starting BulkCheckPresence
0812 06:36:39.424001 (+    39us) tablet.cc:1370] finished BulkCheckPresence
0812 06:36:39.424012 (+    11us) tablet.cc:1372] starting ApplyRowOperation cycle
0812 06:36:39.424209 (+   197us) tablet.cc:1383] finished ApplyRowOperation cycle
0812 06:36:39.424241 (+    32us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0812 06:36:39.424246 (+     5us) write_op.cc:312] APPLY: finished
0812 06:36:39.424318 (+    72us) log.cc:844] Serialized 77 byte log entry
0812 06:36:39.424364 (+    46us) write_op.cc:489] Releasing partition, row and schema locks
0812 06:36:39.424385 (+    21us) write_op.cc:454] Released schema lock
0812 06:36:39.424390 (+     5us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":30,"num_ops":10,"prepare.queue_time_us":1706,"prepare.run_cpu_time_us":601,"prepare.run_wall_time_us":601,"raft.queue_time_us":49,"raft.run_cpu_time_us":143,"raft.run_wall_time_us":143,"replication_time_us":1592431}]]}
I20260812 06:36:39.425716  9019 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:42054 (ReqId={client: 3067f3aef2a44cc4bb7f2ccd6acf83db, seq_no=1646, attempt_no=0}) took 1596 ms. Trace:
I20260812 06:36:39.425765  8932 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:42054 (ReqId={client: 3067f3aef2a44cc4bb7f2ccd6acf83db, seq_no=1647, attempt_no=0}) took 1595 ms. Trace:
I20260812 06:36:39.425836  8932 rpcz_store.cc:276] 0812 06:36:37.830673 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 06:36:37.830718 (+    45us) service_pool.cc:224] Handling call
0812 06:36:39.425756 (+1595038us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0812 06:36:37.832126 (+     0us) write_op.cc:183] PREPARE: starting on tablet 7065bb403e3047dd9a0853c49318ef1f
0812 06:36:37.832196 (+    70us) write_op.cc:432] Acquiring schema lock in shared mode
0812 06:36:37.832199 (+     3us) write_op.cc:435] Acquired schema lock
0812 06:36:37.832201 (+     2us) tablet.cc:662] Decoding operations
0812 06:36:37.832292 (+    91us) write_op.cc:620] Acquiring the partition lock for write op
0812 06:36:37.832297 (+     5us) write_op.cc:641] Partition lock acquired for write op
0812 06:36:37.832300 (+     3us) tablet.cc:685] Acquiring locks for 10 operations
0812 06:36:37.832361 (+    61us) tablet.cc:701] Row locks acquired
0812 06:36:37.832362 (+     1us) write_op.cc:260] PREPARE: finished
0812 06:36:37.832386 (+    24us) write_op.cc:270] Start()
0812 06:36:37.832403 (+    17us) write_op.cc:276] Timestamp: P: 1786516597832366 usec, L: 0
0812 06:36:37.832405 (+     2us) op_driver.cc:348] REPLICATION: starting
0812 06:36:37.832509 (+   104us) log.cc:844] Serialized 696 byte log entry
0812 06:36:39.424037 (+1591528us) op_driver.cc:464] REPLICATION: finished
0812 06:36:39.425383 (+  1346us) write_op.cc:301] APPLY: starting
0812 06:36:39.425392 (+     9us) tablet.cc:1367] starting BulkCheckPresence
0812 06:36:39.425420 (+    28us) tablet.cc:1370] finished BulkCheckPresence
0812 06:36:39.425430 (+    10us) tablet.cc:1372] starting ApplyRowOperation cycle
0812 06:36:39.425596 (+   166us) tablet.cc:1383] finished ApplyRowOperation cycle
0812 06:36:39.425620 (+    24us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0812 06:36:39.425624 (+     4us) write_op.cc:312] APPLY: finished
0812 06:36:39.425672 (+    48us) log.cc:844] Serialized 77 byte log entry
0812 06:36:39.425699 (+    27us) write_op.cc:489] Releasing partition, row and schema locks
0812 06:36:39.425709 (+    10us) write_op.cc:454] Released schema lock
0812 06:36:39.425713 (+     4us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":1323,"num_ops":10,"prepare.queue_time_us":1183,"prepare.run_cpu_time_us":426,"prepare.run_wall_time_us":426,"replication_time_us":1591620,"thread_start_us":32,"threads_started":1}]]}
I20260812 06:36:39.426642  7099 raft_consensus.cc:1240] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [term 13 LEADER]: Rejecting Update request from peer 8d90d61da62549208fe117b4384a469b for earlier term 12. Current term is 13. Ops: []
I20260812 06:36:39.428577  8439 raft_consensus.cc:1240] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [term 13 FOLLOWER]: Rejecting Update request from peer 8d90d61da62549208fe117b4384a469b for earlier term 12. Current term is 13. Ops: [12.1657-12.1661]
I20260812 06:36:39.425773  9019 rpcz_store.cc:276] 0812 06:36:37.828935 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 06:36:37.829322 (+   387us) service_pool.cc:224] Handling call
0812 06:36:39.425692 (+1596370us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0812 06:36:37.831632 (+     0us) write_op.cc:183] PREPARE: starting on tablet 7065bb403e3047dd9a0853c49318ef1f
0812 06:36:37.831707 (+    75us) write_op.cc:432] Acquiring schema lock in shared mode
0812 06:36:37.831710 (+     3us) write_op.cc:435] Acquired schema lock
0812 06:36:37.831712 (+     2us) tablet.cc:662] Decoding operations
0812 06:36:37.831802 (+    90us) write_op.cc:620] Acquiring the partition lock for write op
0812 06:36:37.831813 (+    11us) write_op.cc:641] Partition lock acquired for write op
0812 06:36:37.831816 (+     3us) tablet.cc:685] Acquiring locks for 10 operations
0812 06:36:37.831895 (+    79us) tablet.cc:701] Row locks acquired
0812 06:36:37.831897 (+     2us) write_op.cc:260] PREPARE: finished
0812 06:36:37.831928 (+    31us) write_op.cc:270] Start()
0812 06:36:37.831951 (+    23us) write_op.cc:276] Timestamp: P: 1786516597831908 usec, L: 0
0812 06:36:37.831953 (+     2us) op_driver.cc:348] REPLICATION: starting
0812 06:36:37.832035 (+    82us) log.cc:844] Serialized 696 byte log entry
0812 06:36:39.423916 (+1591881us) op_driver.cc:464] REPLICATION: finished
0812 06:36:39.425276 (+  1360us) write_op.cc:301] APPLY: starting
0812 06:36:39.425289 (+    13us) tablet.cc:1367] starting BulkCheckPresence
0812 06:36:39.425318 (+    29us) tablet.cc:1370] finished BulkCheckPresence
0812 06:36:39.425330 (+    12us) tablet.cc:1372] starting ApplyRowOperation cycle
0812 06:36:39.425508 (+   178us) tablet.cc:1383] finished ApplyRowOperation cycle
0812 06:36:39.425535 (+    27us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0812 06:36:39.425539 (+     4us) write_op.cc:312] APPLY: finished
0812 06:36:39.425593 (+    54us) log.cc:844] Serialized 77 byte log entry
0812 06:36:39.425624 (+    31us) write_op.cc:489] Releasing partition, row and schema locks
0812 06:36:39.425638 (+    14us) write_op.cc:454] Released schema lock
0812 06:36:39.425643 (+     5us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":1320,"num_ops":10,"prepare.queue_time_us":2042,"prepare.run_cpu_time_us":471,"prepare.run_wall_time_us":470,"replication_time_us":1591952,"thread_start_us":77,"threads_started":1}]]}
I20260812 06:36:39.428865  7322 raft_consensus.cc:1240] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08 [term 13 FOLLOWER]: Rejecting Update request from peer 8d90d61da62549208fe117b4384a469b for earlier term 12. Current term is 13. Ops: [12.1582-12.1661]
I20260812 06:36:39.429605  6597 leader_election.cc:304] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 7 responses out of 7 voters: 3 yes votes; 4 no votes. yes voters: 222b20de5a1145c6b8a11df8374e8790, 42895a9bcba64aa484f6509605035d02, b037d17b21f8422fa5e41b70eb4dcd08; no voters: 2df4308c852240859ada53d216ab69ae, 7962f587ccd842ad8a3f6ed9a5d7707e, 8d90d61da62549208fe117b4384a469b, ec6c8d818e744cdbb6b0d15b321010e0
I20260812 06:36:39.430102  9002 raft_consensus.cc:2749] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260812 06:36:39.430694  8961 consensus_queue.cc:1059] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "222b20de5a1145c6b8a11df8374e8790" member_type: VOTER last_known_addr { host: "127.4.177.5" port: 43521 }, Status: INVALID_TERM, Last received: 12.1656, Next index: 1657, Last known committed idx: 1650, Time since last communication: 0.000s
I20260812 06:36:39.430871  8961 consensus_queue.cc:1059] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" member_type: VOTER last_known_addr { host: "127.4.177.7" port: 46299 }, Status: INVALID_TERM, Last received: 12.1581, Next index: 1582, Last known committed idx: 1573, Time since last communication: 0.000s
I20260812 06:36:39.430997  8961 raft_consensus.cc:3055] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b [term 12 LEADER]: Stepping down as leader of term 12
I20260812 06:36:39.431035  8961 raft_consensus.cc:740] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 8d90d61da62549208fe117b4384a469b, State: Running, Role: LEADER
I20260812 06:36:39.431188  8961 consensus_queue.cc:260] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1656, Committed index: 1656, Last appended: 12.1661, Last appended by leader: 1661, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df4308c852240859ada53d216ab69ae" member_type: VOTER last_known_addr { host: "127.4.177.6" port: 34269 } } peers { permanent_uuid: "42895a9bcba64aa484f6509605035d02" member_type: VOTER last_known_addr { host: "127.4.177.4" port: 38181 } } peers { permanent_uuid: "222b20de5a1145c6b8a11df8374e8790" member_type: VOTER last_known_addr { host: "127.4.177.5" port: 43521 } } peers { permanent_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 40897 } } peers { permanent_uuid: "8d90d61da62549208fe117b4384a469b" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 41539 } } peers { permanent_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" member_type: VOTER last_known_addr { host: "127.4.177.7" port: 46299 } } peers { permanent_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 41057 } }
I20260812 06:36:39.431540  8961 raft_consensus.cc:3060] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b [term 12 FOLLOWER]: Advancing to term 13
I20260812 06:36:39.433184  6701 raft_consensus.cc:1240] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [term 13 FOLLOWER]: Rejecting Update request from peer 8d90d61da62549208fe117b4384a469b for earlier term 12. Current term is 13. Ops: []
I20260812 06:36:39.438565  6247 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "222b20de5a1145c6b8a11df8374e8790" candidate_term: 13 candidate_status { last_received { term: 12 index: 1656 } } ignore_live_leader: false dest_uuid: "8d90d61da62549208fe117b4384a469b" is_pre_election: true
I20260812 06:36:39.438730  6247 raft_consensus.cc:2410] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 222b20de5a1145c6b8a11df8374e8790 for term 13 because replica has last-logged OpId of term: 12 index: 1661, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 1656.
I20260812 06:36:39.439066  8327 leader_election.cc:304] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 7 responses out of 7 voters: 3 yes votes; 4 no votes. yes voters: 222b20de5a1145c6b8a11df8374e8790, 7962f587ccd842ad8a3f6ed9a5d7707e, b037d17b21f8422fa5e41b70eb4dcd08; no voters: 2df4308c852240859ada53d216ab69ae, 42895a9bcba64aa484f6509605035d02, 8d90d61da62549208fe117b4384a469b, ec6c8d818e744cdbb6b0d15b321010e0
I20260812 06:36:39.439354  8980 raft_consensus.cc:2749] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260812 06:36:39.440294  9023 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:42054: Illegal state: Cannot assign timestamp to op. Tablet is not in leader mode. Last heard from a leader: 0.018s ago.
W20260812 06:36:39.441216  9023 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:42054: Illegal state: Cannot assign timestamp to op. Tablet is not in leader mode. Last heard from a leader: 0.019s ago.
W20260812 06:36:39.442116  9023 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:42054: Illegal state: Cannot assign timestamp to op. Tablet is not in leader mode. Last heard from a leader: 0.020s ago.
I20260812 06:36:39.452337  6247 raft_consensus.cc:1275] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b [term 13 FOLLOWER]: Refusing update from remote peer 2df4308c852240859ada53d216ab69ae: Log matching property violated. Preceding OpId in replica: term: 12 index: 1661. Preceding OpId from leader: term: 13 index: 1663. (index mismatch)
I20260812 06:36:39.452358  7322 raft_consensus.cc:1275] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08 [term 13 FOLLOWER]: Refusing update from remote peer 2df4308c852240859ada53d216ab69ae: Log matching property violated. Preceding OpId in replica: term: 12 index: 1581. Preceding OpId from leader: term: 13 index: 1663. (index mismatch)
I20260812 06:36:39.452862  8997 consensus_queue.cc:1048] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" member_type: VOTER last_known_addr { host: "127.4.177.7" port: 46299 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1662, Last known committed idx: 1573, Time since last communication: 0.000s
I20260812 06:36:39.453024  9008 consensus_queue.cc:1048] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "8d90d61da62549208fe117b4384a469b" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 41539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1662, Last known committed idx: 1656, Time since last communication: 0.000s
I20260812 06:36:39.455336  8805 raft_consensus.cc:1275] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [term 13 FOLLOWER]: Refusing update from remote peer 2df4308c852240859ada53d216ab69ae: Log matching property violated. Preceding OpId in replica: term: 11 index: 1438. Preceding OpId from leader: term: 13 index: 1663. (index mismatch)
I20260812 06:36:39.455695  6701 raft_consensus.cc:1275] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [term 13 FOLLOWER]: Refusing update from remote peer 2df4308c852240859ada53d216ab69ae: Log matching property violated. Preceding OpId in replica: term: 12 index: 1661. Preceding OpId from leader: term: 13 index: 1663. (index mismatch)
I20260812 06:36:39.456210  9008 consensus_queue.cc:1048] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "42895a9bcba64aa484f6509605035d02" member_type: VOTER last_known_addr { host: "127.4.177.4" port: 38181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1662, Last known committed idx: 1653, Time since last communication: 0.000s
I20260812 06:36:39.456498  9008 consensus_queue.cc:1048] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 40897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1662, Last known committed idx: 1430, Time since last communication: 0.000s
I20260812 06:36:39.456602  9008 consensus_queue.cc:1243] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [LEADER]: Peer 7962f587ccd842ad8a3f6ed9a5d7707e log is divergent from this leader: its last log entry 11.1438 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1430
I20260812 06:36:39.458457  8439 raft_consensus.cc:1275] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [term 13 FOLLOWER]: Refusing update from remote peer 2df4308c852240859ada53d216ab69ae: Log matching property violated. Preceding OpId in replica: term: 12 index: 1656. Preceding OpId from leader: term: 13 index: 1663. (index mismatch)
I20260812 06:36:39.463380  8997 consensus_queue.cc:1048] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "222b20de5a1145c6b8a11df8374e8790" member_type: VOTER last_known_addr { host: "127.4.177.5" port: 43521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1662, Last known committed idx: 1650, Time since last communication: 0.000s
I20260812 06:36:39.464787  8805 pending_rounds.cc:85] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e: Aborting all ops after (but not including) 1430
I20260812 06:36:39.464895  8805 pending_rounds.cc:107] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e: Aborting uncommitted WRITE_OP operation due to leader change: 11.1431
I20260812 06:36:39.465447  8805 pending_rounds.cc:107] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e: Aborting uncommitted WRITE_OP operation due to leader change: 11.1432
I20260812 06:36:39.465668  8805 pending_rounds.cc:107] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e: Aborting uncommitted WRITE_OP operation due to leader change: 11.1433
I20260812 06:36:39.465847  8805 pending_rounds.cc:107] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e: Aborting uncommitted WRITE_OP operation due to leader change: 11.1434
I20260812 06:36:39.466001  8805 pending_rounds.cc:107] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e: Aborting uncommitted WRITE_OP operation due to leader change: 11.1435
I20260812 06:36:39.466154  8805 pending_rounds.cc:107] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e: Aborting uncommitted WRITE_OP operation due to leader change: 11.1436
I20260812 06:36:39.466962  6047 raft_consensus.cc:3060] T 7065bb403e3047dd9a0853c49318ef1f P ec6c8d818e744cdbb6b0d15b321010e0 [term 12 FOLLOWER]: Advancing to term 13
I20260812 06:36:39.468186  6047 raft_consensus.cc:1275] T 7065bb403e3047dd9a0853c49318ef1f P ec6c8d818e744cdbb6b0d15b321010e0 [term 13 FOLLOWER]: Refusing update from remote peer 2df4308c852240859ada53d216ab69ae: Log matching property violated. Preceding OpId in replica: term: 12 index: 1661. Preceding OpId from leader: term: 13 index: 1663. (index mismatch)
I20260812 06:36:39.469378  8997 consensus_queue.cc:1048] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 41057 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1662, Last known committed idx: 1656, Time since last communication: 0.000s
I20260812 06:36:39.471782  8805 pending_rounds.cc:107] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e: Aborting uncommitted WRITE_OP operation due to leader change: 11.1437
I20260812 06:36:39.472049  8805 pending_rounds.cc:107] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e: Aborting uncommitted WRITE_OP operation due to leader change: 11.1438
I20260812 06:36:39.477046  9019 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:42054 (ReqId={client: 3067f3aef2a44cc4bb7f2ccd6acf83db, seq_no=1648, attempt_no=0}) took 1641 ms. Trace:
I20260812 06:36:39.477185  8932 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:42054 (ReqId={client: 3067f3aef2a44cc4bb7f2ccd6acf83db, seq_no=1649, attempt_no=0}) took 1641 ms. Trace:
I20260812 06:36:39.477267  8932 rpcz_store.cc:276] 0812 06:36:37.835994 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 06:36:37.836146 (+   152us) service_pool.cc:224] Handling call
0812 06:36:39.477173 (+1641027us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0812 06:36:37.836723 (+     0us) write_op.cc:183] PREPARE: starting on tablet 7065bb403e3047dd9a0853c49318ef1f
0812 06:36:37.836815 (+    92us) write_op.cc:432] Acquiring schema lock in shared mode
0812 06:36:37.836820 (+     5us) write_op.cc:435] Acquired schema lock
0812 06:36:37.836823 (+     3us) tablet.cc:662] Decoding operations
0812 06:36:37.836978 (+   155us) write_op.cc:620] Acquiring the partition lock for write op
0812 06:36:37.836987 (+     9us) write_op.cc:641] Partition lock acquired for write op
0812 06:36:37.836993 (+     6us) tablet.cc:685] Acquiring locks for 10 operations
0812 06:36:37.837112 (+   119us) tablet.cc:701] Row locks acquired
0812 06:36:37.837115 (+     3us) write_op.cc:260] PREPARE: finished
0812 06:36:37.837146 (+    31us) write_op.cc:270] Start()
0812 06:36:37.837168 (+    22us) write_op.cc:276] Timestamp: P: 1786516597837122 usec, L: 0
0812 06:36:37.837171 (+     3us) op_driver.cc:348] REPLICATION: starting
0812 06:36:37.837288 (+   117us) log.cc:844] Serialized 696 byte log entry
0812 06:36:39.476540 (+1639252us) op_driver.cc:464] REPLICATION: finished
0812 06:36:39.476732 (+   192us) write_op.cc:301] APPLY: starting
0812 06:36:39.476747 (+    15us) tablet.cc:1367] starting BulkCheckPresence
0812 06:36:39.476789 (+    42us) tablet.cc:1370] finished BulkCheckPresence
0812 06:36:39.476801 (+    12us) tablet.cc:1372] starting ApplyRowOperation cycle
0812 06:36:39.476970 (+   169us) tablet.cc:1383] finished ApplyRowOperation cycle
0812 06:36:39.476996 (+    26us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0812 06:36:39.477000 (+     4us) write_op.cc:312] APPLY: finished
0812 06:36:39.477060 (+    60us) log.cc:844] Serialized 77 byte log entry
0812 06:36:39.477096 (+    36us) write_op.cc:489] Releasing partition, row and schema locks
0812 06:36:39.477111 (+    15us) write_op.cc:454] Released schema lock
0812 06:36:39.477116 (+     5us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"spinlock_wait_cycles":20864,"child_traces":[["op",{"apply.queue_time_us":139,"num_ops":10,"prepare.queue_time_us":268,"prepare.run_cpu_time_us":624,"prepare.run_wall_time_us":623,"replication_time_us":1639357}]]}
I20260812 06:36:39.477631  9020 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:42054 (ReqId={client: 3067f3aef2a44cc4bb7f2ccd6acf83db, seq_no=1650, attempt_no=0}) took 1634 ms. Trace:
I20260812 06:36:39.477685  9020 rpcz_store.cc:276] 0812 06:36:37.842985 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 06:36:37.843036 (+    51us) service_pool.cc:224] Handling call
0812 06:36:39.477619 (+1634583us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0812 06:36:37.844458 (+     0us) write_op.cc:183] PREPARE: starting on tablet 7065bb403e3047dd9a0853c49318ef1f
0812 06:36:37.844535 (+    77us) write_op.cc:432] Acquiring schema lock in shared mode
0812 06:36:37.844538 (+     3us) write_op.cc:435] Acquired schema lock
0812 06:36:37.844539 (+     1us) tablet.cc:662] Decoding operations
0812 06:36:37.844635 (+    96us) write_op.cc:620] Acquiring the partition lock for write op
0812 06:36:37.844644 (+     9us) write_op.cc:641] Partition lock acquired for write op
0812 06:36:37.844647 (+     3us) tablet.cc:685] Acquiring locks for 10 operations
0812 06:36:37.844710 (+    63us) tablet.cc:701] Row locks acquired
0812 06:36:37.844711 (+     1us) write_op.cc:260] PREPARE: finished
0812 06:36:37.844736 (+    25us) write_op.cc:270] Start()
0812 06:36:37.844751 (+    15us) write_op.cc:276] Timestamp: P: 1786516597844713 usec, L: 0
0812 06:36:37.844752 (+     1us) op_driver.cc:348] REPLICATION: starting
0812 06:36:37.844868 (+   116us) log.cc:844] Serialized 696 byte log entry
0812 06:36:39.476580 (+1631712us) op_driver.cc:464] REPLICATION: finished
0812 06:36:39.477226 (+   646us) write_op.cc:301] APPLY: starting
0812 06:36:39.477237 (+    11us) tablet.cc:1367] starting BulkCheckPresence
0812 06:36:39.477265 (+    28us) tablet.cc:1370] finished BulkCheckPresence
0812 06:36:39.477278 (+    13us) tablet.cc:1372] starting ApplyRowOperation cycle
0812 06:36:39.477430 (+   152us) tablet.cc:1383] finished ApplyRowOperation cycle
0812 06:36:39.477455 (+    25us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0812 06:36:39.477459 (+     4us) write_op.cc:312] APPLY: finished
0812 06:36:39.477512 (+    53us) log.cc:844] Serialized 77 byte log entry
0812 06:36:39.477555 (+    43us) write_op.cc:489] Releasing partition, row and schema locks
0812 06:36:39.477566 (+    11us) write_op.cc:454] Released schema lock
0812 06:36:39.477570 (+     4us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":614,"num_ops":10,"prepare.queue_time_us":1130,"prepare.run_cpu_time_us":464,"prepare.run_wall_time_us":464,"replication_time_us":1631818}]]}
I20260812 06:36:39.478047  8932 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:42054 (ReqId={client: 3067f3aef2a44cc4bb7f2ccd6acf83db, seq_no=1652, attempt_no=0}) took 1634 ms. Trace:
I20260812 06:36:39.478437  9020 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:42054 (ReqId={client: 3067f3aef2a44cc4bb7f2ccd6acf83db, seq_no=1651, attempt_no=0}) took 1634 ms. Trace:
I20260812 06:36:39.478492  9020 rpcz_store.cc:276] 0812 06:36:37.843641 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 06:36:37.844193 (+   552us) service_pool.cc:224] Handling call
0812 06:36:39.478427 (+1634234us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0812 06:36:37.845845 (+     0us) write_op.cc:183] PREPARE: starting on tablet 7065bb403e3047dd9a0853c49318ef1f
0812 06:36:37.845913 (+    68us) write_op.cc:432] Acquiring schema lock in shared mode
0812 06:36:37.845916 (+     3us) write_op.cc:435] Acquired schema lock
0812 06:36:37.845918 (+     2us) tablet.cc:662] Decoding operations
0812 06:36:37.846018 (+   100us) write_op.cc:620] Acquiring the partition lock for write op
0812 06:36:37.846023 (+     5us) write_op.cc:641] Partition lock acquired for write op
0812 06:36:37.846027 (+     4us) tablet.cc:685] Acquiring locks for 10 operations
0812 06:36:37.846110 (+    83us) tablet.cc:701] Row locks acquired
0812 06:36:37.846112 (+     2us) write_op.cc:260] PREPARE: finished
0812 06:36:37.846138 (+    26us) write_op.cc:270] Start()
0812 06:36:37.846155 (+    17us) write_op.cc:276] Timestamp: P: 1786516597846116 usec, L: 0
0812 06:36:37.846156 (+     1us) op_driver.cc:348] REPLICATION: starting
0812 06:36:37.846309 (+   153us) log.cc:844] Serialized 686 byte log entry
0812 06:36:39.476757 (+1630448us) op_driver.cc:464] REPLICATION: finished
0812 06:36:39.478024 (+  1267us) write_op.cc:301] APPLY: starting
0812 06:36:39.478033 (+     9us) tablet.cc:1367] starting BulkCheckPresence
0812 06:36:39.478058 (+    25us) tablet.cc:1370] finished BulkCheckPresence
0812 06:36:39.478068 (+    10us) tablet.cc:1372] starting ApplyRowOperation cycle
0812 06:36:39.478252 (+   184us) tablet.cc:1383] finished ApplyRowOperation cycle
0812 06:36:39.478278 (+    26us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0812 06:36:39.478282 (+     4us) write_op.cc:312] APPLY: finished
0812 06:36:39.478332 (+    50us) log.cc:844] Serialized 77 byte log entry
0812 06:36:39.478364 (+    32us) write_op.cc:489] Releasing partition, row and schema locks
0812 06:36:39.478374 (+    10us) write_op.cc:454] Released schema lock
0812 06:36:39.478379 (+     5us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":1242,"num_ops":10,"prepare.queue_time_us":1399,"prepare.run_cpu_time_us":510,"prepare.run_wall_time_us":510,"replication_time_us":1630587}]]}
I20260812 06:36:39.477133  9019 rpcz_store.cc:276] 0812 06:36:37.835615 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 06:36:37.835684 (+    69us) service_pool.cc:224] Handling call
0812 06:36:39.477031 (+1641347us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0812 06:36:37.836035 (+     0us) write_op.cc:183] PREPARE: starting on tablet 7065bb403e3047dd9a0853c49318ef1f
0812 06:36:37.836115 (+    80us) write_op.cc:432] Acquiring schema lock in shared mode
0812 06:36:37.836118 (+     3us) write_op.cc:435] Acquired schema lock
0812 06:36:37.836119 (+     1us) tablet.cc:662] Decoding operations
0812 06:36:37.836204 (+    85us) write_op.cc:620] Acquiring the partition lock for write op
0812 06:36:37.836209 (+     5us) write_op.cc:641] Partition lock acquired for write op
0812 06:36:37.836212 (+     3us) tablet.cc:685] Acquiring locks for 10 operations
0812 06:36:37.836272 (+    60us) tablet.cc:701] Row locks acquired
0812 06:36:37.836273 (+     1us) write_op.cc:260] PREPARE: finished
0812 06:36:37.836300 (+    27us) write_op.cc:270] Start()
0812 06:36:37.836318 (+    18us) write_op.cc:276] Timestamp: P: 1786516597836278 usec, L: 0
0812 06:36:37.836320 (+     2us) op_driver.cc:348] REPLICATION: starting
0812 06:36:37.836422 (+   102us) log.cc:844] Serialized 696 byte log entry
0812 06:36:39.476473 (+1640051us) op_driver.cc:464] REPLICATION: finished
0812 06:36:39.476552 (+    79us) write_op.cc:301] APPLY: starting
0812 06:36:39.476565 (+    13us) tablet.cc:1367] starting BulkCheckPresence
0812 06:36:39.476601 (+    36us) tablet.cc:1370] finished BulkCheckPresence
0812 06:36:39.476615 (+    14us) tablet.cc:1372] starting ApplyRowOperation cycle
0812 06:36:39.476821 (+   206us) tablet.cc:1383] finished ApplyRowOperation cycle
0812 06:36:39.476862 (+    41us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0812 06:36:39.476867 (+     5us) write_op.cc:312] APPLY: finished
0812 06:36:39.476922 (+    55us) log.cc:844] Serialized 77 byte log entry
0812 06:36:39.476957 (+    35us) write_op.cc:489] Releasing partition, row and schema locks
0812 06:36:39.476974 (+    17us) write_op.cc:454] Released schema lock
0812 06:36:39.476978 (+     4us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":34,"num_ops":10,"prepare.queue_time_us":52,"prepare.run_cpu_time_us":444,"prepare.run_wall_time_us":656,"replication_time_us":1640130}]]}
I20260812 06:36:39.478107  8932 rpcz_store.cc:276] 0812 06:36:37.843711 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 06:36:37.843754 (+    43us) service_pool.cc:224] Handling call
0812 06:36:39.478038 (+1634284us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0812 06:36:37.844963 (+     0us) write_op.cc:183] PREPARE: starting on tablet 7065bb403e3047dd9a0853c49318ef1f
0812 06:36:37.845392 (+   429us) write_op.cc:432] Acquiring schema lock in shared mode
0812 06:36:37.845396 (+     4us) write_op.cc:435] Acquired schema lock
0812 06:36:37.845397 (+     1us) tablet.cc:662] Decoding operations
0812 06:36:37.845480 (+    83us) write_op.cc:620] Acquiring the partition lock for write op
0812 06:36:37.845485 (+     5us) write_op.cc:641] Partition lock acquired for write op
0812 06:36:37.845487 (+     2us) tablet.cc:685] Acquiring locks for 10 operations
0812 06:36:37.845564 (+    77us) tablet.cc:701] Row locks acquired
0812 06:36:37.845567 (+     3us) write_op.cc:260] PREPARE: finished
0812 06:36:37.845598 (+    31us) write_op.cc:270] Start()
0812 06:36:37.845616 (+    18us) write_op.cc:276] Timestamp: P: 1786516597845575 usec, L: 0
0812 06:36:37.845618 (+     2us) op_driver.cc:348] REPLICATION: starting
0812 06:36:37.845727 (+   109us) log.cc:844] Serialized 696 byte log entry
0812 06:36:39.476611 (+1630884us) op_driver.cc:464] REPLICATION: finished
0812 06:36:39.477634 (+  1023us) write_op.cc:301] APPLY: starting
0812 06:36:39.477645 (+    11us) tablet.cc:1367] starting BulkCheckPresence
0812 06:36:39.477670 (+    25us) tablet.cc:1370] finished BulkCheckPresence
0812 06:36:39.477680 (+    10us) tablet.cc:1372] starting ApplyRowOperation cycle
0812 06:36:39.477854 (+   174us) tablet.cc:1383] finished ApplyRowOperation cycle
0812 06:36:39.477878 (+    24us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0812 06:36:39.477882 (+     4us) write_op.cc:312] APPLY: finished
0812 06:36:39.477928 (+    46us) log.cc:844] Serialized 77 byte log entry
0812 06:36:39.477971 (+    43us) write_op.cc:489] Releasing partition, row and schema locks
0812 06:36:39.477984 (+    13us) write_op.cc:454] Released schema lock
0812 06:36:39.477989 (+     5us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":995,"num_ops":10,"prepare.queue_time_us":971,"prepare.run_cpu_time_us":522,"prepare.run_wall_time_us":859,"replication_time_us":1630984,"spinlock_wait_cycles":80640,"thread_start_us":103,"threads_started":1}]]}
I20260812 06:36:39.729923  9062 mvcc.cc:204] Tried to move back new op lower bound from 7317571991352008704 to 7317571978213814272. Current Snapshot: MvccSnapshot[applied={T|T < 7317571984771944448 or (T in {7317571984775475200,7317571984777691136,7317571991352008704})}]
I20260812 06:36:41.513988  9113 raft_consensus.cc:493] T 7065bb403e3047dd9a0853c49318ef1f P ec6c8d818e744cdbb6b0d15b321010e0 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 2df4308c852240859ada53d216ab69ae)
I20260812 06:36:41.514118  9113 raft_consensus.cc:515] T 7065bb403e3047dd9a0853c49318ef1f P ec6c8d818e744cdbb6b0d15b321010e0 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df4308c852240859ada53d216ab69ae" member_type: VOTER last_known_addr { host: "127.4.177.6" port: 34269 } } peers { permanent_uuid: "42895a9bcba64aa484f6509605035d02" member_type: VOTER last_known_addr { host: "127.4.177.4" port: 38181 } } peers { permanent_uuid: "222b20de5a1145c6b8a11df8374e8790" member_type: VOTER last_known_addr { host: "127.4.177.5" port: 43521 } } peers { permanent_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 40897 } } peers { permanent_uuid: "8d90d61da62549208fe117b4384a469b" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 41539 } } peers { permanent_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" member_type: VOTER last_known_addr { host: "127.4.177.7" port: 46299 } } peers { permanent_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 41057 } }
I20260812 06:36:41.515146  9113 leader_election.cc:290] T 7065bb403e3047dd9a0853c49318ef1f P ec6c8d818e744cdbb6b0d15b321010e0 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 2df4308c852240859ada53d216ab69ae (127.4.177.6:34269), 42895a9bcba64aa484f6509605035d02 (127.4.177.4:38181), 222b20de5a1145c6b8a11df8374e8790 (127.4.177.5:43521), 7962f587ccd842ad8a3f6ed9a5d7707e (127.4.177.3:40897), 8d90d61da62549208fe117b4384a469b (127.4.177.2:41539), b037d17b21f8422fa5e41b70eb4dcd08 (127.4.177.7:46299)
I20260812 06:36:41.515453  6701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" candidate_term: 14 candidate_status { last_received { term: 13 index: 1776 } } ignore_live_leader: false dest_uuid: "42895a9bcba64aa484f6509605035d02" is_pre_election: true
I20260812 06:36:41.515616  7321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" candidate_term: 14 candidate_status { last_received { term: 13 index: 1776 } } ignore_live_leader: false dest_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" is_pre_election: true
I20260812 06:36:41.515815  8437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" candidate_term: 14 candidate_status { last_received { term: 13 index: 1776 } } ignore_live_leader: false dest_uuid: "222b20de5a1145c6b8a11df8374e8790" is_pre_election: true
I20260812 06:36:41.516187  6247 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" candidate_term: 14 candidate_status { last_received { term: 13 index: 1776 } } ignore_live_leader: false dest_uuid: "8d90d61da62549208fe117b4384a469b" is_pre_election: true
I20260812 06:36:41.516585  5945 leader_election.cc:304] T 7065bb403e3047dd9a0853c49318ef1f P ec6c8d818e744cdbb6b0d15b321010e0 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 5 responses out of 7 voters: 1 yes votes; 4 no votes. yes voters: ec6c8d818e744cdbb6b0d15b321010e0; no voters: 222b20de5a1145c6b8a11df8374e8790, 42895a9bcba64aa484f6509605035d02, 8d90d61da62549208fe117b4384a469b, b037d17b21f8422fa5e41b70eb4dcd08
I20260812 06:36:41.516770  9113 raft_consensus.cc:2749] T 7065bb403e3047dd9a0853c49318ef1f P ec6c8d818e744cdbb6b0d15b321010e0 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260812 06:36:41.519498  8805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" candidate_term: 14 candidate_status { last_received { term: 13 index: 1776 } } ignore_live_leader: false dest_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" is_pre_election: true
I20260812 06:36:41.637511  9116 raft_consensus.cc:493] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 2df4308c852240859ada53d216ab69ae)
I20260812 06:36:41.637728  9116 raft_consensus.cc:515] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df4308c852240859ada53d216ab69ae" member_type: VOTER last_known_addr { host: "127.4.177.6" port: 34269 } } peers { permanent_uuid: "42895a9bcba64aa484f6509605035d02" member_type: VOTER last_known_addr { host: "127.4.177.4" port: 38181 } } peers { permanent_uuid: "222b20de5a1145c6b8a11df8374e8790" member_type: VOTER last_known_addr { host: "127.4.177.5" port: 43521 } } peers { permanent_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 40897 } } peers { permanent_uuid: "8d90d61da62549208fe117b4384a469b" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 41539 } } peers { permanent_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" member_type: VOTER last_known_addr { host: "127.4.177.7" port: 46299 } } peers { permanent_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 41057 } }
I20260812 06:36:41.638492  9116 leader_election.cc:290] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 2df4308c852240859ada53d216ab69ae (127.4.177.6:34269), 42895a9bcba64aa484f6509605035d02 (127.4.177.4:38181), 222b20de5a1145c6b8a11df8374e8790 (127.4.177.5:43521), 7962f587ccd842ad8a3f6ed9a5d7707e (127.4.177.3:40897), 8d90d61da62549208fe117b4384a469b (127.4.177.2:41539), ec6c8d818e744cdbb6b0d15b321010e0 (127.4.177.1:41057)
I20260812 06:36:41.639030  6701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" candidate_term: 14 candidate_status { last_received { term: 13 index: 1817 } } ignore_live_leader: false dest_uuid: "42895a9bcba64aa484f6509605035d02" is_pre_election: true
I20260812 06:36:41.639158  8437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" candidate_term: 14 candidate_status { last_received { term: 13 index: 1817 } } ignore_live_leader: false dest_uuid: "222b20de5a1145c6b8a11df8374e8790" is_pre_election: true
I20260812 06:36:41.639194  8805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" candidate_term: 14 candidate_status { last_received { term: 13 index: 1817 } } ignore_live_leader: false dest_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" is_pre_election: true
I20260812 06:36:41.639317  6047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" candidate_term: 14 candidate_status { last_received { term: 13 index: 1817 } } ignore_live_leader: false dest_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" is_pre_election: true
I20260812 06:36:41.639525  6047 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P ec6c8d818e744cdbb6b0d15b321010e0 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b037d17b21f8422fa5e41b70eb4dcd08 in term 13.
I20260812 06:36:41.639775  6247 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" candidate_term: 14 candidate_status { last_received { term: 13 index: 1817 } } ignore_live_leader: false dest_uuid: "8d90d61da62549208fe117b4384a469b" is_pre_election: true
I20260812 06:36:41.640280  7211 leader_election.cc:304] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 6 responses out of 7 voters: 2 yes votes; 4 no votes. yes voters: b037d17b21f8422fa5e41b70eb4dcd08, ec6c8d818e744cdbb6b0d15b321010e0; no voters: 222b20de5a1145c6b8a11df8374e8790, 42895a9bcba64aa484f6509605035d02, 7962f587ccd842ad8a3f6ed9a5d7707e, 8d90d61da62549208fe117b4384a469b
I20260812 06:36:41.640574  9116 raft_consensus.cc:2749] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260812 06:36:41.742066  9118 raft_consensus.cc:493] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 2df4308c852240859ada53d216ab69ae)
I20260812 06:36:41.742272  9118 raft_consensus.cc:515] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df4308c852240859ada53d216ab69ae" member_type: VOTER last_known_addr { host: "127.4.177.6" port: 34269 } } peers { permanent_uuid: "42895a9bcba64aa484f6509605035d02" member_type: VOTER last_known_addr { host: "127.4.177.4" port: 38181 } } peers { permanent_uuid: "222b20de5a1145c6b8a11df8374e8790" member_type: VOTER last_known_addr { host: "127.4.177.5" port: 43521 } } peers { permanent_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 40897 } } peers { permanent_uuid: "8d90d61da62549208fe117b4384a469b" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 41539 } } peers { permanent_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" member_type: VOTER last_known_addr { host: "127.4.177.7" port: 46299 } } peers { permanent_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 41057 } }
I20260812 06:36:41.743124  9118 leader_election.cc:290] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 2df4308c852240859ada53d216ab69ae (127.4.177.6:34269), 42895a9bcba64aa484f6509605035d02 (127.4.177.4:38181), 222b20de5a1145c6b8a11df8374e8790 (127.4.177.5:43521), 7962f587ccd842ad8a3f6ed9a5d7707e (127.4.177.3:40897), b037d17b21f8422fa5e41b70eb4dcd08 (127.4.177.7:46299), ec6c8d818e744cdbb6b0d15b321010e0 (127.4.177.1:41057)
I20260812 06:36:41.743577  7321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "8d90d61da62549208fe117b4384a469b" candidate_term: 14 candidate_status { last_received { term: 13 index: 1858 } } ignore_live_leader: false dest_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" is_pre_election: true
I20260812 06:36:41.743578  6701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "8d90d61da62549208fe117b4384a469b" candidate_term: 14 candidate_status { last_received { term: 13 index: 1858 } } ignore_live_leader: false dest_uuid: "42895a9bcba64aa484f6509605035d02" is_pre_election: true
I20260812 06:36:41.743738  7321 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8d90d61da62549208fe117b4384a469b in term 13.
I20260812 06:36:41.743773  6047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "8d90d61da62549208fe117b4384a469b" candidate_term: 14 candidate_status { last_received { term: 13 index: 1858 } } ignore_live_leader: false dest_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" is_pre_election: true
I20260812 06:36:41.743789  8437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "8d90d61da62549208fe117b4384a469b" candidate_term: 14 candidate_status { last_received { term: 13 index: 1858 } } ignore_live_leader: false dest_uuid: "222b20de5a1145c6b8a11df8374e8790" is_pre_election: true
I20260812 06:36:41.743919  6047 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P ec6c8d818e744cdbb6b0d15b321010e0 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8d90d61da62549208fe117b4384a469b in term 13.
I20260812 06:36:41.744005  8805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "8d90d61da62549208fe117b4384a469b" candidate_term: 14 candidate_status { last_received { term: 13 index: 1858 } } ignore_live_leader: false dest_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" is_pre_election: true
I20260812 06:36:41.863873  9119 raft_consensus.cc:493] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 2df4308c852240859ada53d216ab69ae)
I20260812 06:36:41.864071  9119 raft_consensus.cc:515] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df4308c852240859ada53d216ab69ae" member_type: VOTER last_known_addr { host: "127.4.177.6" port: 34269 } } peers { permanent_uuid: "42895a9bcba64aa484f6509605035d02" member_type: VOTER last_known_addr { host: "127.4.177.4" port: 38181 } } peers { permanent_uuid: "222b20de5a1145c6b8a11df8374e8790" member_type: VOTER last_known_addr { host: "127.4.177.5" port: 43521 } } peers { permanent_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 40897 } } peers { permanent_uuid: "8d90d61da62549208fe117b4384a469b" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 41539 } } peers { permanent_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" member_type: VOTER last_known_addr { host: "127.4.177.7" port: 46299 } } peers { permanent_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 41057 } }
I20260812 06:36:41.864830  9122 raft_consensus.cc:493] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 2df4308c852240859ada53d216ab69ae)
I20260812 06:36:41.864893  9119 leader_election.cc:290] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 2df4308c852240859ada53d216ab69ae (127.4.177.6:34269), 42895a9bcba64aa484f6509605035d02 (127.4.177.4:38181), 7962f587ccd842ad8a3f6ed9a5d7707e (127.4.177.3:40897), 8d90d61da62549208fe117b4384a469b (127.4.177.2:41539), b037d17b21f8422fa5e41b70eb4dcd08 (127.4.177.7:46299), ec6c8d818e744cdbb6b0d15b321010e0 (127.4.177.1:41057)
I20260812 06:36:41.865031  9122 raft_consensus.cc:515] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df4308c852240859ada53d216ab69ae" member_type: VOTER last_known_addr { host: "127.4.177.6" port: 34269 } } peers { permanent_uuid: "42895a9bcba64aa484f6509605035d02" member_type: VOTER last_known_addr { host: "127.4.177.4" port: 38181 } } peers { permanent_uuid: "222b20de5a1145c6b8a11df8374e8790" member_type: VOTER last_known_addr { host: "127.4.177.5" port: 43521 } } peers { permanent_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 40897 } } peers { permanent_uuid: "8d90d61da62549208fe117b4384a469b" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 41539 } } peers { permanent_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" member_type: VOTER last_known_addr { host: "127.4.177.7" port: 46299 } } peers { permanent_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 41057 } }
I20260812 06:36:41.865350  6701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "222b20de5a1145c6b8a11df8374e8790" candidate_term: 14 candidate_status { last_received { term: 13 index: 1910 } } ignore_live_leader: false dest_uuid: "42895a9bcba64aa484f6509605035d02" is_pre_election: true
I20260812 06:36:41.865477  6701 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 222b20de5a1145c6b8a11df8374e8790 in term 13.
I20260812 06:36:41.865546  8805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "222b20de5a1145c6b8a11df8374e8790" candidate_term: 14 candidate_status { last_received { term: 13 index: 1910 } } ignore_live_leader: false dest_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" is_pre_election: true
I20260812 06:36:41.865546  6047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "222b20de5a1145c6b8a11df8374e8790" candidate_term: 14 candidate_status { last_received { term: 13 index: 1910 } } ignore_live_leader: false dest_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" is_pre_election: true
I20260812 06:36:41.865682  6047 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P ec6c8d818e744cdbb6b0d15b321010e0 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 222b20de5a1145c6b8a11df8374e8790 in term 13.
I20260812 06:36:41.865751  6247 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "222b20de5a1145c6b8a11df8374e8790" candidate_term: 14 candidate_status { last_received { term: 13 index: 1910 } } ignore_live_leader: false dest_uuid: "8d90d61da62549208fe117b4384a469b" is_pre_election: true
I20260812 06:36:41.865787  7321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "222b20de5a1145c6b8a11df8374e8790" candidate_term: 14 candidate_status { last_received { term: 13 index: 1910 } } ignore_live_leader: false dest_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" is_pre_election: true
I20260812 06:36:41.865887  6247 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 222b20de5a1145c6b8a11df8374e8790 in term 13.
I20260812 06:36:41.865921  7321 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 222b20de5a1145c6b8a11df8374e8790 in term 13.
I20260812 06:36:41.866530  8327 leader_election.cc:304] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 5 responses out of 7 voters: 4 yes votes; 1 no votes. yes voters: 222b20de5a1145c6b8a11df8374e8790, 42895a9bcba64aa484f6509605035d02, b037d17b21f8422fa5e41b70eb4dcd08, ec6c8d818e744cdbb6b0d15b321010e0; no voters: 7962f587ccd842ad8a3f6ed9a5d7707e
I20260812 06:36:41.866588  7321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "42895a9bcba64aa484f6509605035d02" candidate_term: 14 candidate_status { last_received { term: 13 index: 1910 } } ignore_live_leader: false dest_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" is_pre_election: true
I20260812 06:36:41.866639  9122 leader_election.cc:290] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 2df4308c852240859ada53d216ab69ae (127.4.177.6:34269), 222b20de5a1145c6b8a11df8374e8790 (127.4.177.5:43521), 7962f587ccd842ad8a3f6ed9a5d7707e (127.4.177.3:40897), 8d90d61da62549208fe117b4384a469b (127.4.177.2:41539), b037d17b21f8422fa5e41b70eb4dcd08 (127.4.177.7:46299), ec6c8d818e744cdbb6b0d15b321010e0 (127.4.177.1:41057)
I20260812 06:36:41.866703  7321 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 42895a9bcba64aa484f6509605035d02 in term 13.
I20260812 06:36:41.866786  9119 raft_consensus.cc:2804] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260812 06:36:41.866782  8805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "42895a9bcba64aa484f6509605035d02" candidate_term: 14 candidate_status { last_received { term: 13 index: 1910 } } ignore_live_leader: false dest_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" is_pre_election: true
I20260812 06:36:41.866894  9119 raft_consensus.cc:493] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 2df4308c852240859ada53d216ab69ae)
I20260812 06:36:41.867004  9119 raft_consensus.cc:3060] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [term 13 FOLLOWER]: Advancing to term 14
I20260812 06:36:41.866896  8437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "42895a9bcba64aa484f6509605035d02" candidate_term: 14 candidate_status { last_received { term: 13 index: 1910 } } ignore_live_leader: false dest_uuid: "222b20de5a1145c6b8a11df8374e8790" is_pre_election: true
I20260812 06:36:41.867144  6247 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "42895a9bcba64aa484f6509605035d02" candidate_term: 14 candidate_status { last_received { term: 13 index: 1910 } } ignore_live_leader: false dest_uuid: "8d90d61da62549208fe117b4384a469b" is_pre_election: true
I20260812 06:36:41.867256  6247 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 42895a9bcba64aa484f6509605035d02 in term 13.
I20260812 06:36:41.867496  6047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "42895a9bcba64aa484f6509605035d02" candidate_term: 14 candidate_status { last_received { term: 13 index: 1910 } } ignore_live_leader: false dest_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" is_pre_election: true
I20260812 06:36:41.867619  6047 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P ec6c8d818e744cdbb6b0d15b321010e0 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 42895a9bcba64aa484f6509605035d02 in term 13.
I20260812 06:36:41.867913  6597 leader_election.cc:304] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 5 responses out of 7 voters: 4 yes votes; 1 no votes. yes voters: 42895a9bcba64aa484f6509605035d02, 8d90d61da62549208fe117b4384a469b, b037d17b21f8422fa5e41b70eb4dcd08, ec6c8d818e744cdbb6b0d15b321010e0; no voters: 7962f587ccd842ad8a3f6ed9a5d7707e
I20260812 06:36:41.868093  9122 raft_consensus.cc:2804] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260812 06:36:41.868152  9122 raft_consensus.cc:493] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 2df4308c852240859ada53d216ab69ae)
I20260812 06:36:41.868181  9122 raft_consensus.cc:3060] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [term 13 FOLLOWER]: Advancing to term 14
I20260812 06:36:41.868273  9119 raft_consensus.cc:515] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df4308c852240859ada53d216ab69ae" member_type: VOTER last_known_addr { host: "127.4.177.6" port: 34269 } } peers { permanent_uuid: "42895a9bcba64aa484f6509605035d02" member_type: VOTER last_known_addr { host: "127.4.177.4" port: 38181 } } peers { permanent_uuid: "222b20de5a1145c6b8a11df8374e8790" member_type: VOTER last_known_addr { host: "127.4.177.5" port: 43521 } } peers { permanent_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 40897 } } peers { permanent_uuid: "8d90d61da62549208fe117b4384a469b" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 41539 } } peers { permanent_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" member_type: VOTER last_known_addr { host: "127.4.177.7" port: 46299 } } peers { permanent_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 41057 } }
I20260812 06:36:41.868511  8437 raft_consensus.cc:2393] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 42895a9bcba64aa484f6509605035d02 in current term 14: Already voted for candidate 222b20de5a1145c6b8a11df8374e8790 in this term.
I20260812 06:36:41.868986  9119 leader_election.cc:290] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [CANDIDATE]: Term 14 election: Requested vote from peers 2df4308c852240859ada53d216ab69ae (127.4.177.6:34269), 42895a9bcba64aa484f6509605035d02 (127.4.177.4:38181), 7962f587ccd842ad8a3f6ed9a5d7707e (127.4.177.3:40897), 8d90d61da62549208fe117b4384a469b (127.4.177.2:41539), b037d17b21f8422fa5e41b70eb4dcd08 (127.4.177.7:46299), ec6c8d818e744cdbb6b0d15b321010e0 (127.4.177.1:41057)
I20260812 06:36:41.869354  6247 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "222b20de5a1145c6b8a11df8374e8790" candidate_term: 14 candidate_status { last_received { term: 13 index: 1910 } } ignore_live_leader: false dest_uuid: "8d90d61da62549208fe117b4384a469b"
I20260812 06:36:41.869365  6047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "222b20de5a1145c6b8a11df8374e8790" candidate_term: 14 candidate_status { last_received { term: 13 index: 1910 } } ignore_live_leader: false dest_uuid: "ec6c8d818e744cdbb6b0d15b321010e0"
I20260812 06:36:41.869472  6247 raft_consensus.cc:3060] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b [term 13 FOLLOWER]: Advancing to term 14
I20260812 06:36:41.869468  6047 raft_consensus.cc:3060] T 7065bb403e3047dd9a0853c49318ef1f P ec6c8d818e744cdbb6b0d15b321010e0 [term 13 FOLLOWER]: Advancing to term 14
I20260812 06:36:41.869441  9122 raft_consensus.cc:515] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df4308c852240859ada53d216ab69ae" member_type: VOTER last_known_addr { host: "127.4.177.6" port: 34269 } } peers { permanent_uuid: "42895a9bcba64aa484f6509605035d02" member_type: VOTER last_known_addr { host: "127.4.177.4" port: 38181 } } peers { permanent_uuid: "222b20de5a1145c6b8a11df8374e8790" member_type: VOTER last_known_addr { host: "127.4.177.5" port: 43521 } } peers { permanent_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 40897 } } peers { permanent_uuid: "8d90d61da62549208fe117b4384a469b" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 41539 } } peers { permanent_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" member_type: VOTER last_known_addr { host: "127.4.177.7" port: 46299 } } peers { permanent_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 41057 } }
I20260812 06:36:41.869642  6701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "222b20de5a1145c6b8a11df8374e8790" candidate_term: 14 candidate_status { last_received { term: 13 index: 1910 } } ignore_live_leader: false dest_uuid: "42895a9bcba64aa484f6509605035d02"
I20260812 06:36:41.869768  6701 raft_consensus.cc:2393] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 222b20de5a1145c6b8a11df8374e8790 in current term 14: Already voted for candidate 42895a9bcba64aa484f6509605035d02 in this term.
I20260812 06:36:41.869962  8805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "222b20de5a1145c6b8a11df8374e8790" candidate_term: 14 candidate_status { last_received { term: 13 index: 1910 } } ignore_live_leader: false dest_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e"
I20260812 06:36:41.870072  9122 leader_election.cc:290] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [CANDIDATE]: Term 14 election: Requested vote from peers 2df4308c852240859ada53d216ab69ae (127.4.177.6:34269), 222b20de5a1145c6b8a11df8374e8790 (127.4.177.5:43521), 7962f587ccd842ad8a3f6ed9a5d7707e (127.4.177.3:40897), 8d90d61da62549208fe117b4384a469b (127.4.177.2:41539), b037d17b21f8422fa5e41b70eb4dcd08 (127.4.177.7:46299), ec6c8d818e744cdbb6b0d15b321010e0 (127.4.177.1:41057)
I20260812 06:36:41.870615  6247 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 222b20de5a1145c6b8a11df8374e8790 in term 14.
I20260812 06:36:41.870625  6047 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P ec6c8d818e744cdbb6b0d15b321010e0 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 222b20de5a1145c6b8a11df8374e8790 in term 14.
I20260812 06:36:41.870827  8805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "42895a9bcba64aa484f6509605035d02" candidate_term: 14 candidate_status { last_received { term: 13 index: 1910 } } ignore_live_leader: false dest_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e"
I20260812 06:36:41.870967  8437 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "42895a9bcba64aa484f6509605035d02" candidate_term: 14 candidate_status { last_received { term: 13 index: 1910 } } ignore_live_leader: false dest_uuid: "222b20de5a1145c6b8a11df8374e8790"
I20260812 06:36:41.871040  6047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "42895a9bcba64aa484f6509605035d02" candidate_term: 14 candidate_status { last_received { term: 13 index: 1910 } } ignore_live_leader: false dest_uuid: "ec6c8d818e744cdbb6b0d15b321010e0"
I20260812 06:36:41.871105  8437 raft_consensus.cc:2393] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 42895a9bcba64aa484f6509605035d02 in current term 14: Already voted for candidate 222b20de5a1145c6b8a11df8374e8790 in this term.
I20260812 06:36:41.871155  6047 raft_consensus.cc:2393] T 7065bb403e3047dd9a0853c49318ef1f P ec6c8d818e744cdbb6b0d15b321010e0 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 42895a9bcba64aa484f6509605035d02 in current term 14: Already voted for candidate 222b20de5a1145c6b8a11df8374e8790 in this term.
I20260812 06:36:41.871426  7322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "222b20de5a1145c6b8a11df8374e8790" candidate_term: 14 candidate_status { last_received { term: 13 index: 1910 } } ignore_live_leader: false dest_uuid: "b037d17b21f8422fa5e41b70eb4dcd08"
I20260812 06:36:41.871426  7321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "42895a9bcba64aa484f6509605035d02" candidate_term: 14 candidate_status { last_received { term: 13 index: 1910 } } ignore_live_leader: false dest_uuid: "b037d17b21f8422fa5e41b70eb4dcd08"
I20260812 06:36:41.871584  7322 raft_consensus.cc:3060] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08 [term 13 FOLLOWER]: Advancing to term 14
I20260812 06:36:41.871649  6247 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "42895a9bcba64aa484f6509605035d02" candidate_term: 14 candidate_status { last_received { term: 13 index: 1910 } } ignore_live_leader: false dest_uuid: "8d90d61da62549208fe117b4384a469b"
I20260812 06:36:41.871773  6247 raft_consensus.cc:2393] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 42895a9bcba64aa484f6509605035d02 in current term 14: Already voted for candidate 222b20de5a1145c6b8a11df8374e8790 in this term.
I20260812 06:36:41.872027  6595 leader_election.cc:304] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 5 responses out of 7 voters: 1 yes votes; 4 no votes. yes voters: 42895a9bcba64aa484f6509605035d02; no voters: 222b20de5a1145c6b8a11df8374e8790, 7962f587ccd842ad8a3f6ed9a5d7707e, b037d17b21f8422fa5e41b70eb4dcd08, ec6c8d818e744cdbb6b0d15b321010e0
I20260812 06:36:41.872188  9122 raft_consensus.cc:2749] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260812 06:36:41.872613  7322 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 222b20de5a1145c6b8a11df8374e8790 in term 14.
I20260812 06:36:41.872931  8323 leader_election.cc:304] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 6 responses out of 7 voters: 4 yes votes; 2 no votes. yes voters: 222b20de5a1145c6b8a11df8374e8790, 8d90d61da62549208fe117b4384a469b, b037d17b21f8422fa5e41b70eb4dcd08, ec6c8d818e744cdbb6b0d15b321010e0; no voters: 42895a9bcba64aa484f6509605035d02, 7962f587ccd842ad8a3f6ed9a5d7707e
I20260812 06:36:41.873163  9119 raft_consensus.cc:2804] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [term 14 FOLLOWER]: Leader election won for term 14
I20260812 06:36:41.873375  9119 raft_consensus.cc:697] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [term 14 LEADER]: Becoming Leader. State: Replica: 222b20de5a1145c6b8a11df8374e8790, State: Running, Role: LEADER
I20260812 06:36:41.873728  9119 consensus_queue.cc:237] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1902, Committed index: 1902, Last appended: 13.1910, Last appended by leader: 1910, Current term: 14, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df4308c852240859ada53d216ab69ae" member_type: VOTER last_known_addr { host: "127.4.177.6" port: 34269 } } peers { permanent_uuid: "42895a9bcba64aa484f6509605035d02" member_type: VOTER last_known_addr { host: "127.4.177.4" port: 38181 } } peers { permanent_uuid: "222b20de5a1145c6b8a11df8374e8790" member_type: VOTER last_known_addr { host: "127.4.177.5" port: 43521 } } peers { permanent_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 40897 } } peers { permanent_uuid: "8d90d61da62549208fe117b4384a469b" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 41539 } } peers { permanent_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" member_type: VOTER last_known_addr { host: "127.4.177.7" port: 46299 } } peers { permanent_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 41057 } }
I20260812 06:36:41.876376  5812 catalog_manager.cc:5719] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 reported cstate change: term changed from 13 to 14, leader changed from 2df4308c852240859ada53d216ab69ae (127.4.177.6) to 222b20de5a1145c6b8a11df8374e8790 (127.4.177.5). New cstate: current_term: 14 leader_uuid: "222b20de5a1145c6b8a11df8374e8790" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df4308c852240859ada53d216ab69ae" member_type: VOTER last_known_addr { host: "127.4.177.6" port: 34269 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "42895a9bcba64aa484f6509605035d02" member_type: VOTER last_known_addr { host: "127.4.177.4" port: 38181 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "222b20de5a1145c6b8a11df8374e8790" member_type: VOTER last_known_addr { host: "127.4.177.5" port: 43521 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 40897 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8d90d61da62549208fe117b4384a469b" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 41539 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" member_type: VOTER last_known_addr { host: "127.4.177.7" port: 46299 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 41057 } health_report { overall_health: UNKNOWN } } }
I20260812 06:36:41.952982  7099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" candidate_term: 14 candidate_status { last_received { term: 13 index: 1817 } } ignore_live_leader: false dest_uuid: "2df4308c852240859ada53d216ab69ae" is_pre_election: true
W20260812 06:36:41.953166  7020 connection.cc:256] RPC call timeout handler was delayed by -1.0527334581129253s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1s for a response
I20260812 06:36:41.954804  7100 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "42895a9bcba64aa484f6509605035d02" candidate_term: 14 candidate_status { last_received { term: 13 index: 1910 } } ignore_live_leader: false dest_uuid: "2df4308c852240859ada53d216ab69ae" is_pre_election: true
I20260812 06:36:41.955054  7101 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "42895a9bcba64aa484f6509605035d02" candidate_term: 14 candidate_status { last_received { term: 13 index: 1910 } } ignore_live_leader: false dest_uuid: "2df4308c852240859ada53d216ab69ae"
I20260812 06:36:41.955233  7102 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "222b20de5a1145c6b8a11df8374e8790" candidate_term: 14 candidate_status { last_received { term: 13 index: 1910 } } ignore_live_leader: false dest_uuid: "2df4308c852240859ada53d216ab69ae" is_pre_election: true
I20260812 06:36:41.955451  7104 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "8d90d61da62549208fe117b4384a469b" candidate_term: 14 candidate_status { last_received { term: 13 index: 1858 } } ignore_live_leader: false dest_uuid: "2df4308c852240859ada53d216ab69ae" is_pre_election: true
I20260812 06:36:41.955663  7105 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "222b20de5a1145c6b8a11df8374e8790" candidate_term: 14 candidate_status { last_received { term: 13 index: 1910 } } ignore_live_leader: false dest_uuid: "2df4308c852240859ada53d216ab69ae"
I20260812 06:36:41.955892  7106 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" candidate_term: 14 candidate_status { last_received { term: 13 index: 1776 } } ignore_live_leader: false dest_uuid: "2df4308c852240859ada53d216ab69ae" is_pre_election: true
I20260812 06:36:41.955989  8437 raft_consensus.cc:1240] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [term 14 LEADER]: Rejecting Update request from peer 2df4308c852240859ada53d216ab69ae for earlier term 13. Current term is 14. Ops: []
I20260812 06:36:41.957461  6150 leader_election.cc:304] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 7 responses out of 7 voters: 3 yes votes; 4 no votes. yes voters: 8d90d61da62549208fe117b4384a469b, b037d17b21f8422fa5e41b70eb4dcd08, ec6c8d818e744cdbb6b0d15b321010e0; no voters: 222b20de5a1145c6b8a11df8374e8790, 2df4308c852240859ada53d216ab69ae, 42895a9bcba64aa484f6509605035d02, 7962f587ccd842ad8a3f6ed9a5d7707e
I20260812 06:36:41.957679  9065 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:50492 (ReqId={client: 3067f3aef2a44cc4bb7f2ccd6acf83db, seq_no=1893, attempt_no=0}) took 1609 ms. Trace:
I20260812 06:36:41.957794  9118 raft_consensus.cc:2749] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260812 06:36:41.957733  9065 rpcz_store.cc:276] 0812 06:36:40.348004 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 06:36:40.348089 (+    85us) service_pool.cc:224] Handling call
0812 06:36:41.957653 (+1609564us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0812 06:36:40.349861 (+     0us) write_op.cc:183] PREPARE: starting on tablet 7065bb403e3047dd9a0853c49318ef1f
0812 06:36:40.349936 (+    75us) write_op.cc:432] Acquiring schema lock in shared mode
0812 06:36:40.349939 (+     3us) write_op.cc:435] Acquired schema lock
0812 06:36:40.349941 (+     2us) tablet.cc:662] Decoding operations
0812 06:36:40.350036 (+    95us) write_op.cc:620] Acquiring the partition lock for write op
0812 06:36:40.350041 (+     5us) write_op.cc:641] Partition lock acquired for write op
0812 06:36:40.350044 (+     3us) tablet.cc:685] Acquiring locks for 10 operations
0812 06:36:40.350113 (+    69us) tablet.cc:701] Row locks acquired
0812 06:36:40.350115 (+     2us) write_op.cc:260] PREPARE: finished
0812 06:36:40.350140 (+    25us) write_op.cc:270] Start()
0812 06:36:40.350160 (+    20us) write_op.cc:276] Timestamp: P: 1786516600350079 usec, L: 0
0812 06:36:40.350163 (+     3us) op_driver.cc:348] REPLICATION: starting
0812 06:36:40.350297 (+   134us) log.cc:844] Serialized 696 byte log entry
0812 06:36:41.956824 (+1606527us) op_driver.cc:464] REPLICATION: finished
0812 06:36:41.956988 (+   164us) write_op.cc:301] APPLY: starting
0812 06:36:41.957008 (+    20us) tablet.cc:1367] starting BulkCheckPresence
0812 06:36:41.957055 (+    47us) tablet.cc:1370] finished BulkCheckPresence
0812 06:36:41.957070 (+    15us) tablet.cc:1372] starting ApplyRowOperation cycle
0812 06:36:41.957307 (+   237us) tablet.cc:1383] finished ApplyRowOperation cycle
0812 06:36:41.957342 (+    35us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0812 06:36:41.957347 (+     5us) write_op.cc:312] APPLY: finished
0812 06:36:41.957422 (+    75us) log.cc:844] Serialized 77 byte log entry
0812 06:36:41.957567 (+   145us) write_op.cc:489] Releasing partition, row and schema locks
0812 06:36:41.957583 (+    16us) write_op.cc:454] Released schema lock
0812 06:36:41.957589 (+     6us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":85,"num_ops":10,"prepare.queue_time_us":1506,"prepare.run_cpu_time_us":483,"prepare.run_wall_time_us":654,"replication_time_us":1606630,"thread_start_us":69,"threads_started":1}]]}
I20260812 06:36:41.958618  9065 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:50492 (ReqId={client: 3067f3aef2a44cc4bb7f2ccd6acf83db, seq_no=1894, attempt_no=0}) took 1610 ms. Trace:
I20260812 06:36:41.958673  9065 rpcz_store.cc:276] 0812 06:36:40.348563 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 06:36:40.349282 (+   719us) service_pool.cc:224] Handling call
0812 06:36:41.958609 (+1609327us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0812 06:36:40.350543 (+     0us) write_op.cc:183] PREPARE: starting on tablet 7065bb403e3047dd9a0853c49318ef1f
0812 06:36:40.350614 (+    71us) write_op.cc:432] Acquiring schema lock in shared mode
0812 06:36:40.350618 (+     4us) write_op.cc:435] Acquired schema lock
0812 06:36:40.350619 (+     1us) tablet.cc:662] Decoding operations
0812 06:36:40.350708 (+    89us) write_op.cc:620] Acquiring the partition lock for write op
0812 06:36:40.350714 (+     6us) write_op.cc:641] Partition lock acquired for write op
0812 06:36:40.350717 (+     3us) tablet.cc:685] Acquiring locks for 10 operations
0812 06:36:40.350788 (+    71us) tablet.cc:701] Row locks acquired
0812 06:36:40.350789 (+     1us) write_op.cc:260] PREPARE: finished
0812 06:36:40.350814 (+    25us) write_op.cc:270] Start()
0812 06:36:40.350832 (+    18us) write_op.cc:276] Timestamp: P: 1786516600350752 usec, L: 0
0812 06:36:40.350834 (+     2us) op_driver.cc:348] REPLICATION: starting
0812 06:36:40.350925 (+    91us) log.cc:844] Serialized 696 byte log entry
0812 06:36:41.956894 (+1605969us) op_driver.cc:464] REPLICATION: finished
0812 06:36:41.958171 (+  1277us) write_op.cc:301] APPLY: starting
0812 06:36:41.958182 (+    11us) tablet.cc:1367] starting BulkCheckPresence
0812 06:36:41.958230 (+    48us) tablet.cc:1370] finished BulkCheckPresence
0812 06:36:41.958241 (+    11us) tablet.cc:1372] starting ApplyRowOperation cycle
0812 06:36:41.958435 (+   194us) tablet.cc:1383] finished ApplyRowOperation cycle
0812 06:36:41.958461 (+    26us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0812 06:36:41.958466 (+     5us) write_op.cc:312] APPLY: finished
0812 06:36:41.958514 (+    48us) log.cc:844] Serialized 77 byte log entry
0812 06:36:41.958546 (+    32us) write_op.cc:489] Releasing partition, row and schema locks
0812 06:36:41.958556 (+    10us) write_op.cc:454] Released schema lock
0812 06:36:41.958561 (+     5us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":1246,"num_ops":10,"prepare.queue_time_us":1024,"prepare.run_cpu_time_us":431,"prepare.run_wall_time_us":619,"replication_time_us":1606043,"thread_start_us":93,"threads_started":1}]]}
I20260812 06:36:41.959059  9131 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:50492 (ReqId={client: 3067f3aef2a44cc4bb7f2ccd6acf83db, seq_no=1895, attempt_no=0}) took 1609 ms. Trace:
I20260812 06:36:41.959120  9131 rpcz_store.cc:276] 0812 06:36:40.349714 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 06:36:40.349763 (+    49us) service_pool.cc:224] Handling call
0812 06:36:41.959034 (+1609271us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0812 06:36:40.351196 (+     0us) write_op.cc:183] PREPARE: starting on tablet 7065bb403e3047dd9a0853c49318ef1f
0812 06:36:40.351277 (+    81us) write_op.cc:432] Acquiring schema lock in shared mode
0812 06:36:40.351280 (+     3us) write_op.cc:435] Acquired schema lock
0812 06:36:40.351282 (+     2us) tablet.cc:662] Decoding operations
0812 06:36:40.351366 (+    84us) write_op.cc:620] Acquiring the partition lock for write op
0812 06:36:40.351371 (+     5us) write_op.cc:641] Partition lock acquired for write op
0812 06:36:40.351374 (+     3us) tablet.cc:685] Acquiring locks for 10 operations
0812 06:36:40.351443 (+    69us) tablet.cc:701] Row locks acquired
0812 06:36:40.351446 (+     3us) write_op.cc:260] PREPARE: finished
0812 06:36:40.351478 (+    32us) write_op.cc:270] Start()
0812 06:36:40.351498 (+    20us) write_op.cc:276] Timestamp: P: 1786516600351414 usec, L: 0
0812 06:36:40.351500 (+     2us) op_driver.cc:348] REPLICATION: starting
0812 06:36:40.351602 (+   102us) log.cc:844] Serialized 696 byte log entry
0812 06:36:41.957049 (+1605447us) op_driver.cc:464] REPLICATION: finished
0812 06:36:41.958629 (+  1580us) write_op.cc:301] APPLY: starting
0812 06:36:41.958643 (+    14us) tablet.cc:1367] starting BulkCheckPresence
0812 06:36:41.958671 (+    28us) tablet.cc:1370] finished BulkCheckPresence
0812 06:36:41.958683 (+    12us) tablet.cc:1372] starting ApplyRowOperation cycle
0812 06:36:41.958860 (+   177us) tablet.cc:1383] finished ApplyRowOperation cycle
0812 06:36:41.958884 (+    24us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0812 06:36:41.958889 (+     5us) write_op.cc:312] APPLY: finished
0812 06:36:41.958940 (+    51us) log.cc:844] Serialized 77 byte log entry
0812 06:36:41.958970 (+    30us) write_op.cc:489] Releasing partition, row and schema locks
0812 06:36:41.958983 (+    13us) write_op.cc:454] Released schema lock
0812 06:36:41.958987 (+     4us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":1546,"num_ops":10,"prepare.queue_time_us":1202,"prepare.run_cpu_time_us":461,"prepare.run_wall_time_us":2070,"replication_time_us":1605536,"thread_start_us":50,"threads_started":1}]]}
I20260812 06:36:41.959409  9065 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:50492 (ReqId={client: 3067f3aef2a44cc4bb7f2ccd6acf83db, seq_no=1896, attempt_no=0}) took 1609 ms. Trace:
I20260812 06:36:41.959462  9065 rpcz_store.cc:276] 0812 06:36:40.349822 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 06:36:40.349864 (+    42us) service_pool.cc:224] Handling call
0812 06:36:41.959399 (+1609535us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0812 06:36:40.353294 (+     0us) write_op.cc:183] PREPARE: starting on tablet 7065bb403e3047dd9a0853c49318ef1f
0812 06:36:40.353363 (+    69us) write_op.cc:432] Acquiring schema lock in shared mode
0812 06:36:40.353366 (+     3us) write_op.cc:435] Acquired schema lock
0812 06:36:40.353368 (+     2us) tablet.cc:662] Decoding operations
0812 06:36:40.353453 (+    85us) write_op.cc:620] Acquiring the partition lock for write op
0812 06:36:40.353459 (+     6us) write_op.cc:641] Partition lock acquired for write op
0812 06:36:40.353462 (+     3us) tablet.cc:685] Acquiring locks for 10 operations
0812 06:36:40.353520 (+    58us) tablet.cc:701] Row locks acquired
0812 06:36:40.353522 (+     2us) write_op.cc:260] PREPARE: finished
0812 06:36:40.353547 (+    25us) write_op.cc:270] Start()
0812 06:36:40.353564 (+    17us) write_op.cc:276] Timestamp: P: 1786516600353485 usec, L: 0
0812 06:36:40.353566 (+     2us) op_driver.cc:348] REPLICATION: starting
0812 06:36:40.353666 (+   100us) log.cc:844] Serialized 696 byte log entry
0812 06:36:41.957133 (+1603467us) op_driver.cc:464] REPLICATION: finished
0812 06:36:41.958979 (+  1846us) write_op.cc:301] APPLY: starting
0812 06:36:41.958988 (+     9us) tablet.cc:1367] starting BulkCheckPresence
0812 06:36:41.959014 (+    26us) tablet.cc:1370] finished BulkCheckPresence
0812 06:36:41.959024 (+    10us) tablet.cc:1372] starting ApplyRowOperation cycle
0812 06:36:41.959213 (+   189us) tablet.cc:1383] finished ApplyRowOperation cycle
0812 06:36:41.959236 (+    23us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0812 06:36:41.959240 (+     4us) write_op.cc:312] APPLY: finished
0812 06:36:41.959284 (+    44us) log.cc:844] Serialized 77 byte log entry
0812 06:36:41.959316 (+    32us) write_op.cc:489] Releasing partition, row and schema locks
0812 06:36:41.959328 (+    12us) write_op.cc:454] Released schema lock
0812 06:36:41.959332 (+     4us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":1823,"num_ops":10,"prepare.queue_time_us":3179,"prepare.run_cpu_time_us":420,"prepare.run_wall_time_us":600,"replication_time_us":1603556,"thread_start_us":35,"threads_started":1}]]}
I20260812 06:36:41.959822  9131 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:50492 (ReqId={client: 3067f3aef2a44cc4bb7f2ccd6acf83db, seq_no=1897, attempt_no=0}) took 1608 ms. Trace:
I20260812 06:36:41.959884  9131 rpcz_store.cc:276] 0812 06:36:40.350914 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 06:36:40.350957 (+    43us) service_pool.cc:224] Handling call
0812 06:36:41.959813 (+1608856us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0812 06:36:40.353920 (+     0us) write_op.cc:183] PREPARE: starting on tablet 7065bb403e3047dd9a0853c49318ef1f
0812 06:36:40.353994 (+    74us) write_op.cc:432] Acquiring schema lock in shared mode
0812 06:36:40.353997 (+     3us) write_op.cc:435] Acquired schema lock
0812 06:36:40.353998 (+     1us) tablet.cc:662] Decoding operations
0812 06:36:40.354098 (+   100us) write_op.cc:620] Acquiring the partition lock for write op
0812 06:36:40.354104 (+     6us) write_op.cc:641] Partition lock acquired for write op
0812 06:36:40.354107 (+     3us) tablet.cc:685] Acquiring locks for 10 operations
0812 06:36:40.354165 (+    58us) tablet.cc:701] Row locks acquired
0812 06:36:40.354166 (+     1us) write_op.cc:260] PREPARE: finished
0812 06:36:40.354193 (+    27us) write_op.cc:270] Start()
0812 06:36:40.354228 (+    35us) write_op.cc:276] Timestamp: P: 1786516600354131 usec, L: 0
0812 06:36:40.354230 (+     2us) op_driver.cc:348] REPLICATION: starting
0812 06:36:40.354327 (+    97us) log.cc:844] Serialized 696 byte log entry
0812 06:36:41.957198 (+1602871us) op_driver.cc:464] REPLICATION: finished
0812 06:36:41.959435 (+  2237us) write_op.cc:301] APPLY: starting
0812 06:36:41.959443 (+     8us) tablet.cc:1367] starting BulkCheckPresence
0812 06:36:41.959470 (+    27us) tablet.cc:1370] finished BulkCheckPresence
0812 06:36:41.959480 (+    10us) tablet.cc:1372] starting ApplyRowOperation cycle
0812 06:36:41.959636 (+   156us) tablet.cc:1383] finished ApplyRowOperation cycle
0812 06:36:41.959659 (+    23us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0812 06:36:41.959664 (+     5us) write_op.cc:312] APPLY: finished
0812 06:36:41.959710 (+    46us) log.cc:844] Serialized 77 byte log entry
0812 06:36:41.959757 (+    47us) write_op.cc:489] Releasing partition, row and schema locks
0812 06:36:41.959767 (+    10us) write_op.cc:454] Released schema lock
0812 06:36:41.959771 (+     4us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":2215,"num_ops":10,"prepare.queue_time_us":2709,"prepare.run_cpu_time_us":455,"prepare.run_wall_time_us":632,"replication_time_us":1602958}]]}
I20260812 06:36:41.960049  7322 raft_consensus.cc:1240] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08 [term 14 FOLLOWER]: Rejecting Update request from peer 2df4308c852240859ada53d216ab69ae for earlier term 13. Current term is 14. Ops: [13.1818-13.1910]
I20260812 06:36:41.960127  9065 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:50492 (ReqId={client: 3067f3aef2a44cc4bb7f2ccd6acf83db, seq_no=1898, attempt_no=0}) took 1608 ms. Trace:
I20260812 06:36:41.960173  9065 rpcz_store.cc:276] 0812 06:36:40.351560 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 06:36:40.351604 (+    44us) service_pool.cc:224] Handling call
0812 06:36:41.960119 (+1608515us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0812 06:36:40.354577 (+     0us) write_op.cc:183] PREPARE: starting on tablet 7065bb403e3047dd9a0853c49318ef1f
0812 06:36:40.354646 (+    69us) write_op.cc:432] Acquiring schema lock in shared mode
0812 06:36:40.354649 (+     3us) write_op.cc:435] Acquired schema lock
0812 06:36:40.354651 (+     2us) tablet.cc:662] Decoding operations
0812 06:36:40.354727 (+    76us) write_op.cc:620] Acquiring the partition lock for write op
0812 06:36:40.354733 (+     6us) write_op.cc:641] Partition lock acquired for write op
0812 06:36:40.354736 (+     3us) tablet.cc:685] Acquiring locks for 10 operations
0812 06:36:40.354796 (+    60us) tablet.cc:701] Row locks acquired
0812 06:36:40.354798 (+     2us) write_op.cc:260] PREPARE: finished
0812 06:36:40.354823 (+    25us) write_op.cc:270] Start()
0812 06:36:40.354839 (+    16us) write_op.cc:276] Timestamp: P: 1786516600354761 usec, L: 0
0812 06:36:40.354841 (+     2us) op_driver.cc:348] REPLICATION: starting
0812 06:36:40.354936 (+    95us) log.cc:844] Serialized 696 byte log entry
0812 06:36:41.957219 (+1602283us) op_driver.cc:464] REPLICATION: finished
0812 06:36:41.959740 (+  2521us) write_op.cc:301] APPLY: starting
0812 06:36:41.959748 (+     8us) tablet.cc:1367] starting BulkCheckPresence
0812 06:36:41.959773 (+    25us) tablet.cc:1370] finished BulkCheckPresence
0812 06:36:41.959783 (+    10us) tablet.cc:1372] starting ApplyRowOperation cycle
0812 06:36:41.959945 (+   162us) tablet.cc:1383] finished ApplyRowOperation cycle
0812 06:36:41.959969 (+    24us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0812 06:36:41.959973 (+     4us) write_op.cc:312] APPLY: finished
0812 06:36:41.960018 (+    45us) log.cc:844] Serialized 77 byte log entry
0812 06:36:41.960049 (+    31us) write_op.cc:489] Releasing partition, row and schema locks
0812 06:36:41.960059 (+    10us) write_op.cc:454] Released schema lock
0812 06:36:41.960066 (+     7us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":2500,"num_ops":10,"prepare.queue_time_us":2717,"prepare.run_cpu_time_us":407,"prepare.run_wall_time_us":612,"replication_time_us":1602370}]]}
I20260812 06:36:41.962301  9094 consensus_queue.cc:1059] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" member_type: VOTER last_known_addr { host: "127.4.177.7" port: 46299 }, Status: INVALID_TERM, Last received: 13.1817, Next index: 1818, Last known committed idx: 1816, Time since last communication: 0.000s
I20260812 06:36:41.962469  9094 consensus_queue.cc:1059] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "222b20de5a1145c6b8a11df8374e8790" member_type: VOTER last_known_addr { host: "127.4.177.5" port: 43521 }, Status: INVALID_TERM, Last received: 13.1910, Next index: 1911, Last known committed idx: 1902, Time since last communication: 0.000s
I20260812 06:36:41.962572  9094 consensus_queue.cc:1243] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [LEADER]: Peer 222b20de5a1145c6b8a11df8374e8790 log is divergent from this leader: its last log entry 14.1911 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1902
I20260812 06:36:41.962637  9094 raft_consensus.cc:3055] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [term 13 LEADER]: Stepping down as leader of term 13
I20260812 06:36:41.962673  9094 raft_consensus.cc:740] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 2df4308c852240859ada53d216ab69ae, State: Running, Role: LEADER
I20260812 06:36:41.962817  9094 consensus_queue.cc:260] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1908, Committed index: 1908, Last appended: 13.1910, Last appended by leader: 1910, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df4308c852240859ada53d216ab69ae" member_type: VOTER last_known_addr { host: "127.4.177.6" port: 34269 } } peers { permanent_uuid: "42895a9bcba64aa484f6509605035d02" member_type: VOTER last_known_addr { host: "127.4.177.4" port: 38181 } } peers { permanent_uuid: "222b20de5a1145c6b8a11df8374e8790" member_type: VOTER last_known_addr { host: "127.4.177.5" port: 43521 } } peers { permanent_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 40897 } } peers { permanent_uuid: "8d90d61da62549208fe117b4384a469b" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 41539 } } peers { permanent_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" member_type: VOTER last_known_addr { host: "127.4.177.7" port: 46299 } } peers { permanent_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 41057 } }
I20260812 06:36:41.963038  9094 raft_consensus.cc:3060] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [term 13 FOLLOWER]: Advancing to term 14
I20260812 06:36:41.963954  6701 raft_consensus.cc:1240] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [term 14 FOLLOWER]: Rejecting Update request from peer 2df4308c852240859ada53d216ab69ae for earlier term 13. Current term is 14. Ops: []
I20260812 06:36:41.964179  6247 raft_consensus.cc:1240] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b [term 14 FOLLOWER]: Rejecting Update request from peer 2df4308c852240859ada53d216ab69ae for earlier term 13. Current term is 14. Ops: [13.1859-13.1910]
W20260812 06:36:41.965440  7060 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50492: Illegal state: replica 2df4308c852240859ada53d216ab69ae is not leader of this config: current role FOLLOWER
W20260812 06:36:41.965637  7054 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50492: Illegal state: replica 2df4308c852240859ada53d216ab69ae is not leader of this config: current role FOLLOWER
W20260812 06:36:41.965637  7059 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50492: Illegal state: replica 2df4308c852240859ada53d216ab69ae is not leader of this config: current role FOLLOWER
W20260812 06:36:41.965878  7061 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50492: Illegal state: replica 2df4308c852240859ada53d216ab69ae is not leader of this config: current role FOLLOWER
W20260812 06:36:41.965927  7051 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50492: Illegal state: replica 2df4308c852240859ada53d216ab69ae is not leader of this config: current role FOLLOWER
W20260812 06:36:41.965456  7063 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50492: Illegal state: replica 2df4308c852240859ada53d216ab69ae is not leader of this config: current role FOLLOWER
I20260812 06:36:41.966761  6047 raft_consensus.cc:1240] T 7065bb403e3047dd9a0853c49318ef1f P ec6c8d818e744cdbb6b0d15b321010e0 [term 14 FOLLOWER]: Rejecting Update request from peer 2df4308c852240859ada53d216ab69ae for earlier term 13. Current term is 14. Ops: [13.1777-13.1910]
W20260812 06:36:41.968365  6626 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:42408: Illegal state: replica 42895a9bcba64aa484f6509605035d02 is not leader of this config: current role FOLLOWER
W20260812 06:36:41.970621  6626 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:42408: Illegal state: replica 42895a9bcba64aa484f6509605035d02 is not leader of this config: current role FOLLOWER
W20260812 06:36:41.971581  6626 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:42408: Illegal state: replica 42895a9bcba64aa484f6509605035d02 is not leader of this config: current role FOLLOWER
W20260812 06:36:41.971663  6621 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:42408: Illegal state: replica 42895a9bcba64aa484f6509605035d02 is not leader of this config: current role FOLLOWER
W20260812 06:36:41.972575  6621 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:42408: Illegal state: replica 42895a9bcba64aa484f6509605035d02 is not leader of this config: current role FOLLOWER
W20260812 06:36:41.972711  6626 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:42408: Illegal state: replica 42895a9bcba64aa484f6509605035d02 is not leader of this config: current role FOLLOWER
W20260812 06:36:41.974999  6184 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:42054: Illegal state: replica 8d90d61da62549208fe117b4384a469b is not leader of this config: current role FOLLOWER
W20260812 06:36:41.975008  6182 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:42054: Illegal state: replica 8d90d61da62549208fe117b4384a469b is not leader of this config: current role FOLLOWER
W20260812 06:36:41.975063  6179 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:42054: Illegal state: replica 8d90d61da62549208fe117b4384a469b is not leader of this config: current role FOLLOWER
W20260812 06:36:41.977413  6184 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:42054: Illegal state: replica 8d90d61da62549208fe117b4384a469b is not leader of this config: current role FOLLOWER
W20260812 06:36:41.977520  6182 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:42054: Illegal state: replica 8d90d61da62549208fe117b4384a469b is not leader of this config: current role FOLLOWER
W20260812 06:36:41.977586  6179 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:42054: Illegal state: replica 8d90d61da62549208fe117b4384a469b is not leader of this config: current role FOLLOWER
W20260812 06:36:41.979954  7249 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:58454: Illegal state: replica b037d17b21f8422fa5e41b70eb4dcd08 is not leader of this config: current role FOLLOWER
W20260812 06:36:41.979969  7254 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:58454: Illegal state: replica b037d17b21f8422fa5e41b70eb4dcd08 is not leader of this config: current role FOLLOWER
W20260812 06:36:41.982983  7249 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:58454: Illegal state: replica b037d17b21f8422fa5e41b70eb4dcd08 is not leader of this config: current role FOLLOWER
W20260812 06:36:41.984100  7249 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:58454: Illegal state: replica b037d17b21f8422fa5e41b70eb4dcd08 is not leader of this config: current role FOLLOWER
W20260812 06:36:41.985474  7249 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:58454: Illegal state: replica b037d17b21f8422fa5e41b70eb4dcd08 is not leader of this config: current role FOLLOWER
W20260812 06:36:41.986660  5985 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:42334: Illegal state: replica ec6c8d818e744cdbb6b0d15b321010e0 is not leader of this config: current role FOLLOWER
W20260812 06:36:41.986829  5984 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:42334: Illegal state: replica ec6c8d818e744cdbb6b0d15b321010e0 is not leader of this config: current role FOLLOWER
W20260812 06:36:41.987222  5988 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:42334: Illegal state: replica ec6c8d818e744cdbb6b0d15b321010e0 is not leader of this config: current role FOLLOWER
W20260812 06:36:41.989809  5988 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:42334: Illegal state: replica ec6c8d818e744cdbb6b0d15b321010e0 is not leader of this config: current role FOLLOWER
W20260812 06:36:41.990003  5984 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:42334: Illegal state: replica ec6c8d818e744cdbb6b0d15b321010e0 is not leader of this config: current role FOLLOWER
W20260812 06:36:41.990909  5984 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:42334: Illegal state: replica ec6c8d818e744cdbb6b0d15b321010e0 is not leader of this config: current role FOLLOWER
W20260812 06:36:41.997638  7249 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:58454: Illegal state: replica b037d17b21f8422fa5e41b70eb4dcd08 is not leader of this config: current role FOLLOWER
I20260812 06:36:42.004067  7322 raft_consensus.cc:1275] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08 [term 14 FOLLOWER]: Refusing update from remote peer 222b20de5a1145c6b8a11df8374e8790: Log matching property violated. Preceding OpId in replica: term: 13 index: 1817. Preceding OpId from leader: term: 14 index: 1912. (index mismatch)
I20260812 06:36:42.004165  6247 raft_consensus.cc:1275] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b [term 14 FOLLOWER]: Refusing update from remote peer 222b20de5a1145c6b8a11df8374e8790: Log matching property violated. Preceding OpId in replica: term: 13 index: 1858. Preceding OpId from leader: term: 14 index: 1912. (index mismatch)
I20260812 06:36:42.004142  8805 raft_consensus.cc:3060] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [term 13 FOLLOWER]: Advancing to term 14
I20260812 06:36:42.004416  6047 raft_consensus.cc:1275] T 7065bb403e3047dd9a0853c49318ef1f P ec6c8d818e744cdbb6b0d15b321010e0 [term 14 FOLLOWER]: Refusing update from remote peer 222b20de5a1145c6b8a11df8374e8790: Log matching property violated. Preceding OpId in replica: term: 13 index: 1776. Preceding OpId from leader: term: 14 index: 1912. (index mismatch)
I20260812 06:36:42.004544  7109 raft_consensus.cc:1275] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [term 14 FOLLOWER]: Refusing update from remote peer 222b20de5a1145c6b8a11df8374e8790: Log matching property violated. Preceding OpId in replica: term: 13 index: 1910. Preceding OpId from leader: term: 14 index: 1912. (index mismatch)
I20260812 06:36:42.004971  9163 consensus_queue.cc:1048] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8d90d61da62549208fe117b4384a469b" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 41539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1911, Last known committed idx: 1854, Time since last communication: 0.000s
I20260812 06:36:42.005471  6701 raft_consensus.cc:1275] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [term 14 FOLLOWER]: Refusing update from remote peer 222b20de5a1145c6b8a11df8374e8790: Log matching property violated. Preceding OpId in replica: term: 13 index: 1910. Preceding OpId from leader: term: 14 index: 1912. (index mismatch)
I20260812 06:36:42.005492  9158 consensus_queue.cc:1048] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 41057 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1911, Last known committed idx: 1776, Time since last communication: 0.000s
I20260812 06:36:42.005625  8805 raft_consensus.cc:1275] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [term 14 FOLLOWER]: Refusing update from remote peer 222b20de5a1145c6b8a11df8374e8790: Log matching property violated. Preceding OpId in replica: term: 13 index: 1910. Preceding OpId from leader: term: 14 index: 1912. (index mismatch)
I20260812 06:36:42.005802  9159 consensus_queue.cc:1048] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" member_type: VOTER last_known_addr { host: "127.4.177.7" port: 46299 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1911, Last known committed idx: 1816, Time since last communication: 0.000s
I20260812 06:36:42.006489  9119 consensus_queue.cc:1048] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 40897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1911, Last known committed idx: 1908, Time since last communication: 0.000s
I20260812 06:36:42.007647  9158 consensus_queue.cc:1048] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [LEADER]: Connected to new peer: Peer: permanent_uuid: "42895a9bcba64aa484f6509605035d02" member_type: VOTER last_known_addr { host: "127.4.177.4" port: 38181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1911, Last known committed idx: 1902, Time since last communication: 0.000s
I20260812 06:36:42.007879  9123 consensus_queue.cc:1048] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2df4308c852240859ada53d216ab69ae" member_type: VOTER last_known_addr { host: "127.4.177.6" port: 34269 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1911, Last known committed idx: 1908, Time since last communication: 0.000s
I20260812 06:36:42.045516  9135 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:50492 (ReqId={client: 3067f3aef2a44cc4bb7f2ccd6acf83db, seq_no=1900, attempt_no=0}) took 1684 ms. Trace:
I20260812 06:36:42.045523  9131 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:50492 (ReqId={client: 3067f3aef2a44cc4bb7f2ccd6acf83db, seq_no=1899, attempt_no=0}) took 1685 ms. Trace:
I20260812 06:36:42.045681  9131 rpcz_store.cc:276] 0812 06:36:40.360520 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 06:36:40.360592 (+    72us) service_pool.cc:224] Handling call
0812 06:36:42.045510 (+1684918us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0812 06:36:40.361211 (+     0us) write_op.cc:183] PREPARE: starting on tablet 7065bb403e3047dd9a0853c49318ef1f
0812 06:36:40.361283 (+    72us) write_op.cc:432] Acquiring schema lock in shared mode
0812 06:36:40.361285 (+     2us) write_op.cc:435] Acquired schema lock
0812 06:36:40.361287 (+     2us) tablet.cc:662] Decoding operations
0812 06:36:40.361371 (+    84us) write_op.cc:620] Acquiring the partition lock for write op
0812 06:36:40.361377 (+     6us) write_op.cc:641] Partition lock acquired for write op
0812 06:36:40.361379 (+     2us) tablet.cc:685] Acquiring locks for 10 operations
0812 06:36:40.361447 (+    68us) tablet.cc:701] Row locks acquired
0812 06:36:40.361448 (+     1us) write_op.cc:260] PREPARE: finished
0812 06:36:40.361474 (+    26us) write_op.cc:270] Start()
0812 06:36:40.361491 (+    17us) write_op.cc:276] Timestamp: P: 1786516600361411 usec, L: 0
0812 06:36:40.361493 (+     2us) op_driver.cc:348] REPLICATION: starting
0812 06:36:40.361609 (+   116us) log.cc:844] Serialized 686 byte log entry
0812 06:36:42.044614 (+1683005us) op_driver.cc:464] REPLICATION: finished
0812 06:36:42.044947 (+   333us) write_op.cc:301] APPLY: starting
0812 06:36:42.044975 (+    28us) tablet.cc:1367] starting BulkCheckPresence
0812 06:36:42.045007 (+    32us) tablet.cc:1370] finished BulkCheckPresence
0812 06:36:42.045022 (+    15us) tablet.cc:1372] starting ApplyRowOperation cycle
0812 06:36:42.045252 (+   230us) tablet.cc:1383] finished ApplyRowOperation cycle
0812 06:36:42.045300 (+    48us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0812 06:36:42.045305 (+     5us) write_op.cc:312] APPLY: finished
0812 06:36:42.045373 (+    68us) log.cc:844] Serialized 77 byte log entry
0812 06:36:42.045418 (+    45us) write_op.cc:489] Releasing partition, row and schema locks
0812 06:36:42.045431 (+    13us) write_op.cc:454] Released schema lock
0812 06:36:42.045436 (+     5us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":271,"num_ops":10,"prepare.queue_time_us":322,"prepare.run_cpu_time_us":453,"prepare.run_wall_time_us":453,"replication_time_us":1683098,"spinlock_wait_cycles":19456}]]}
I20260812 06:36:42.045662  9135 rpcz_store.cc:276] 0812 06:36:40.360774 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 06:36:40.360814 (+    40us) service_pool.cc:224] Handling call
0812 06:36:42.045478 (+1684664us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0812 06:36:40.361685 (+     0us) write_op.cc:183] PREPARE: starting on tablet 7065bb403e3047dd9a0853c49318ef1f
0812 06:36:40.361743 (+    58us) write_op.cc:432] Acquiring schema lock in shared mode
0812 06:36:40.361746 (+     3us) write_op.cc:435] Acquired schema lock
0812 06:36:40.361747 (+     1us) tablet.cc:662] Decoding operations
0812 06:36:40.361822 (+    75us) write_op.cc:620] Acquiring the partition lock for write op
0812 06:36:40.361826 (+     4us) write_op.cc:641] Partition lock acquired for write op
0812 06:36:40.361829 (+     3us) tablet.cc:685] Acquiring locks for 10 operations
0812 06:36:40.361885 (+    56us) tablet.cc:701] Row locks acquired
0812 06:36:40.361887 (+     2us) write_op.cc:260] PREPARE: finished
0812 06:36:40.361912 (+    25us) write_op.cc:270] Start()
0812 06:36:40.361931 (+    19us) write_op.cc:276] Timestamp: P: 1786516600361848 usec, L: 0
0812 06:36:40.361933 (+     2us) op_driver.cc:348] REPLICATION: starting
0812 06:36:40.362067 (+   134us) log.cc:844] Serialized 686 byte log entry
0812 06:36:42.044678 (+1682611us) op_driver.cc:464] REPLICATION: finished
0812 06:36:42.045059 (+   381us) write_op.cc:301] APPLY: starting
0812 06:36:42.045070 (+    11us) tablet.cc:1367] starting BulkCheckPresence
0812 06:36:42.045097 (+    27us) tablet.cc:1370] finished BulkCheckPresence
0812 06:36:42.045112 (+    15us) tablet.cc:1372] starting ApplyRowOperation cycle
0812 06:36:42.045284 (+   172us) tablet.cc:1383] finished ApplyRowOperation cycle
0812 06:36:42.045309 (+    25us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0812 06:36:42.045313 (+     4us) write_op.cc:312] APPLY: finished
0812 06:36:42.045364 (+    51us) log.cc:844] Serialized 77 byte log entry
0812 06:36:42.045397 (+    33us) write_op.cc:489] Releasing partition, row and schema locks
0812 06:36:42.045413 (+    16us) write_op.cc:454] Released schema lock
0812 06:36:42.045417 (+     4us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":346,"num_ops":10,"prepare.queue_time_us":627,"prepare.run_cpu_time_us":434,"prepare.run_wall_time_us":433,"replication_time_us":1682733}]]}
I20260812 06:36:42.898351  4804 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskCnZu03/build/debug/bin/kudu with pid 8298
W20260812 06:36:42.914268  7401 meta_cache.cc:302] tablet 7065bb403e3047dd9a0853c49318ef1f: replica 222b20de5a1145c6b8a11df8374e8790 (127.4.177.5:43521) has failed: Network error: recv got EOF from 127.4.177.5:43521 (error 108)
W20260812 06:36:42.918946  7207 connection.cc:466] server connection from 127.4.177.5:56745 torn down before Call kudu.consensus.ConsensusService.UpdateConsensus from 127.4.177.5:56745 (request call id 47) could send its response
W20260812 06:36:42.925269  6147 connection.cc:466] server connection from 127.4.177.5:49847 torn down before Call kudu.consensus.ConsensusService.UpdateConsensus from 127.4.177.5:49847 (request call id 40) could send its response
W20260812 06:36:42.945827  8694 connection.cc:466] server connection from 127.4.177.5:33851 torn down before Call kudu.consensus.ConsensusService.UpdateConsensus from 127.4.177.5:33851 (request call id 71) could send its response
I20260812 06:36:43.748076  9211 raft_consensus.cc:493] T 7065bb403e3047dd9a0853c49318ef1f P ec6c8d818e744cdbb6b0d15b321010e0 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 222b20de5a1145c6b8a11df8374e8790)
I20260812 06:36:43.748248  9211 raft_consensus.cc:515] T 7065bb403e3047dd9a0853c49318ef1f P ec6c8d818e744cdbb6b0d15b321010e0 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df4308c852240859ada53d216ab69ae" member_type: VOTER last_known_addr { host: "127.4.177.6" port: 34269 } } peers { permanent_uuid: "42895a9bcba64aa484f6509605035d02" member_type: VOTER last_known_addr { host: "127.4.177.4" port: 38181 } } peers { permanent_uuid: "222b20de5a1145c6b8a11df8374e8790" member_type: VOTER last_known_addr { host: "127.4.177.5" port: 43521 } } peers { permanent_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 40897 } } peers { permanent_uuid: "8d90d61da62549208fe117b4384a469b" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 41539 } } peers { permanent_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" member_type: VOTER last_known_addr { host: "127.4.177.7" port: 46299 } } peers { permanent_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 41057 } }
I20260812 06:36:43.748884  9211 leader_election.cc:290] T 7065bb403e3047dd9a0853c49318ef1f P ec6c8d818e744cdbb6b0d15b321010e0 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 2df4308c852240859ada53d216ab69ae (127.4.177.6:34269), 42895a9bcba64aa484f6509605035d02 (127.4.177.4:38181), 222b20de5a1145c6b8a11df8374e8790 (127.4.177.5:43521), 7962f587ccd842ad8a3f6ed9a5d7707e (127.4.177.3:40897), 8d90d61da62549208fe117b4384a469b (127.4.177.2:41539), b037d17b21f8422fa5e41b70eb4dcd08 (127.4.177.7:46299)
I20260812 06:36:43.749410  7322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" candidate_term: 15 candidate_status { last_received { term: 14 index: 1949 } } ignore_live_leader: false dest_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" is_pre_election: true
I20260812 06:36:43.749433  7107 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" candidate_term: 15 candidate_status { last_received { term: 14 index: 1949 } } ignore_live_leader: false dest_uuid: "2df4308c852240859ada53d216ab69ae" is_pre_election: true
W20260812 06:36:43.749833  5945 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.4.177.5:43521: connect: Connection refused (error 111) [suppressed 1 similar messages]
I20260812 06:36:43.749909  6701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" candidate_term: 15 candidate_status { last_received { term: 14 index: 1949 } } ignore_live_leader: false dest_uuid: "42895a9bcba64aa484f6509605035d02" is_pre_election: true
I20260812 06:36:43.750298  8805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" candidate_term: 15 candidate_status { last_received { term: 14 index: 1949 } } ignore_live_leader: false dest_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" is_pre_election: true
I20260812 06:36:43.750312  6238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" candidate_term: 15 candidate_status { last_received { term: 14 index: 1949 } } ignore_live_leader: false dest_uuid: "8d90d61da62549208fe117b4384a469b" is_pre_election: true
I20260812 06:36:43.750814  5945 leader_election.cc:304] T 7065bb403e3047dd9a0853c49318ef1f P ec6c8d818e744cdbb6b0d15b321010e0 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 5 responses out of 7 voters: 1 yes votes; 4 no votes. yes voters: ec6c8d818e744cdbb6b0d15b321010e0; no voters: 2df4308c852240859ada53d216ab69ae, 42895a9bcba64aa484f6509605035d02, 7962f587ccd842ad8a3f6ed9a5d7707e, b037d17b21f8422fa5e41b70eb4dcd08
I20260812 06:36:43.751019  9211 raft_consensus.cc:2749] T 7065bb403e3047dd9a0853c49318ef1f P ec6c8d818e744cdbb6b0d15b321010e0 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260812 06:36:43.751179  5945 leader_election.cc:336] T 7065bb403e3047dd9a0853c49318ef1f P ec6c8d818e744cdbb6b0d15b321010e0 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 222b20de5a1145c6b8a11df8374e8790 (127.4.177.5:43521): Network error: Client connection negotiation failed: client connection to 127.4.177.5:43521: connect: Connection refused (error 111)
I20260812 06:36:43.953547  9214 raft_consensus.cc:493] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 222b20de5a1145c6b8a11df8374e8790)
I20260812 06:36:43.953720  9214 raft_consensus.cc:515] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df4308c852240859ada53d216ab69ae" member_type: VOTER last_known_addr { host: "127.4.177.6" port: 34269 } } peers { permanent_uuid: "42895a9bcba64aa484f6509605035d02" member_type: VOTER last_known_addr { host: "127.4.177.4" port: 38181 } } peers { permanent_uuid: "222b20de5a1145c6b8a11df8374e8790" member_type: VOTER last_known_addr { host: "127.4.177.5" port: 43521 } } peers { permanent_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 40897 } } peers { permanent_uuid: "8d90d61da62549208fe117b4384a469b" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 41539 } } peers { permanent_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" member_type: VOTER last_known_addr { host: "127.4.177.7" port: 46299 } } peers { permanent_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 41057 } }
I20260812 06:36:43.954540  9214 leader_election.cc:290] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 2df4308c852240859ada53d216ab69ae (127.4.177.6:34269), 42895a9bcba64aa484f6509605035d02 (127.4.177.4:38181), 222b20de5a1145c6b8a11df8374e8790 (127.4.177.5:43521), 7962f587ccd842ad8a3f6ed9a5d7707e (127.4.177.3:40897), b037d17b21f8422fa5e41b70eb4dcd08 (127.4.177.7:46299), ec6c8d818e744cdbb6b0d15b321010e0 (127.4.177.1:41057)
I20260812 06:36:43.954979  7107 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "8d90d61da62549208fe117b4384a469b" candidate_term: 15 candidate_status { last_received { term: 14 index: 2019 } } ignore_live_leader: false dest_uuid: "2df4308c852240859ada53d216ab69ae" is_pre_election: true
I20260812 06:36:43.955327  8805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "8d90d61da62549208fe117b4384a469b" candidate_term: 15 candidate_status { last_received { term: 14 index: 2019 } } ignore_live_leader: false dest_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" is_pre_election: true
W20260812 06:36:43.955482  6150 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.4.177.5:43521: connect: Connection refused (error 111) [suppressed 2 similar messages]
I20260812 06:36:43.955773  6701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "8d90d61da62549208fe117b4384a469b" candidate_term: 15 candidate_status { last_received { term: 14 index: 2019 } } ignore_live_leader: false dest_uuid: "42895a9bcba64aa484f6509605035d02" is_pre_election: true
I20260812 06:36:43.955981  7322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "8d90d61da62549208fe117b4384a469b" candidate_term: 15 candidate_status { last_received { term: 14 index: 2019 } } ignore_live_leader: false dest_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" is_pre_election: true
I20260812 06:36:43.956296  6047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "8d90d61da62549208fe117b4384a469b" candidate_term: 15 candidate_status { last_received { term: 14 index: 2019 } } ignore_live_leader: false dest_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" is_pre_election: true
I20260812 06:36:43.956440  6147 leader_election.cc:304] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 5 responses out of 7 voters: 1 yes votes; 4 no votes. yes voters: 8d90d61da62549208fe117b4384a469b; no voters: 2df4308c852240859ada53d216ab69ae, 42895a9bcba64aa484f6509605035d02, 7962f587ccd842ad8a3f6ed9a5d7707e, b037d17b21f8422fa5e41b70eb4dcd08
I20260812 06:36:43.956498  6047 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P ec6c8d818e744cdbb6b0d15b321010e0 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8d90d61da62549208fe117b4384a469b in term 14.
I20260812 06:36:43.956640  9214 raft_consensus.cc:2749] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260812 06:36:43.957228  6150 leader_election.cc:336] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 222b20de5a1145c6b8a11df8374e8790 (127.4.177.5:43521): Network error: Client connection negotiation failed: client connection to 127.4.177.5:43521: connect: Connection refused (error 111)
I20260812 06:36:44.071628  9218 raft_consensus.cc:493] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 222b20de5a1145c6b8a11df8374e8790)
I20260812 06:36:44.071777  9218 raft_consensus.cc:515] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df4308c852240859ada53d216ab69ae" member_type: VOTER last_known_addr { host: "127.4.177.6" port: 34269 } } peers { permanent_uuid: "42895a9bcba64aa484f6509605035d02" member_type: VOTER last_known_addr { host: "127.4.177.4" port: 38181 } } peers { permanent_uuid: "222b20de5a1145c6b8a11df8374e8790" member_type: VOTER last_known_addr { host: "127.4.177.5" port: 43521 } } peers { permanent_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 40897 } } peers { permanent_uuid: "8d90d61da62549208fe117b4384a469b" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 41539 } } peers { permanent_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" member_type: VOTER last_known_addr { host: "127.4.177.7" port: 46299 } } peers { permanent_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 41057 } }
I20260812 06:36:44.072574  9218 leader_election.cc:290] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 42895a9bcba64aa484f6509605035d02 (127.4.177.4:38181), 222b20de5a1145c6b8a11df8374e8790 (127.4.177.5:43521), 7962f587ccd842ad8a3f6ed9a5d7707e (127.4.177.3:40897), 8d90d61da62549208fe117b4384a469b (127.4.177.2:41539), b037d17b21f8422fa5e41b70eb4dcd08 (127.4.177.7:46299), ec6c8d818e744cdbb6b0d15b321010e0 (127.4.177.1:41057)
I20260812 06:36:44.072990  9219 raft_consensus.cc:493] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 222b20de5a1145c6b8a11df8374e8790)
I20260812 06:36:44.073024  7322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "2df4308c852240859ada53d216ab69ae" candidate_term: 15 candidate_status { last_received { term: 14 index: 2073 } } ignore_live_leader: false dest_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" is_pre_election: true
I20260812 06:36:44.073025  6701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "2df4308c852240859ada53d216ab69ae" candidate_term: 15 candidate_status { last_received { term: 14 index: 2073 } } ignore_live_leader: false dest_uuid: "42895a9bcba64aa484f6509605035d02" is_pre_election: true
I20260812 06:36:44.073201  9219 raft_consensus.cc:515] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df4308c852240859ada53d216ab69ae" member_type: VOTER last_known_addr { host: "127.4.177.6" port: 34269 } } peers { permanent_uuid: "42895a9bcba64aa484f6509605035d02" member_type: VOTER last_known_addr { host: "127.4.177.4" port: 38181 } } peers { permanent_uuid: "222b20de5a1145c6b8a11df8374e8790" member_type: VOTER last_known_addr { host: "127.4.177.5" port: 43521 } } peers { permanent_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 40897 } } peers { permanent_uuid: "8d90d61da62549208fe117b4384a469b" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 41539 } } peers { permanent_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" member_type: VOTER last_known_addr { host: "127.4.177.7" port: 46299 } } peers { permanent_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 41057 } }
I20260812 06:36:44.073443  6701 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2df4308c852240859ada53d216ab69ae in term 14.
I20260812 06:36:44.073724  8805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "2df4308c852240859ada53d216ab69ae" candidate_term: 15 candidate_status { last_received { term: 14 index: 2073 } } ignore_live_leader: false dest_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" is_pre_election: true
W20260812 06:36:44.073814  7020 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.4.177.5:43521: connect: Connection refused (error 111) [suppressed 1 similar messages]
I20260812 06:36:44.073930  6047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "2df4308c852240859ada53d216ab69ae" candidate_term: 15 candidate_status { last_received { term: 14 index: 2073 } } ignore_live_leader: false dest_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" is_pre_election: true
I20260812 06:36:44.074079  6047 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P ec6c8d818e744cdbb6b0d15b321010e0 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2df4308c852240859ada53d216ab69ae in term 14.
I20260812 06:36:44.074299  7107 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "42895a9bcba64aa484f6509605035d02" candidate_term: 15 candidate_status { last_received { term: 14 index: 2073 } } ignore_live_leader: false dest_uuid: "2df4308c852240859ada53d216ab69ae" is_pre_election: true
I20260812 06:36:44.074426  7107 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 42895a9bcba64aa484f6509605035d02 in term 14.
I20260812 06:36:44.074702  6238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "2df4308c852240859ada53d216ab69ae" candidate_term: 15 candidate_status { last_received { term: 14 index: 2073 } } ignore_live_leader: false dest_uuid: "8d90d61da62549208fe117b4384a469b" is_pre_election: true
I20260812 06:36:44.074846  6238 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2df4308c852240859ada53d216ab69ae in term 14.
I20260812 06:36:44.074839  8805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "42895a9bcba64aa484f6509605035d02" candidate_term: 15 candidate_status { last_received { term: 14 index: 2073 } } ignore_live_leader: false dest_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" is_pre_election: true
I20260812 06:36:44.075147  6238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "42895a9bcba64aa484f6509605035d02" candidate_term: 15 candidate_status { last_received { term: 14 index: 2073 } } ignore_live_leader: false dest_uuid: "8d90d61da62549208fe117b4384a469b" is_pre_election: true
I20260812 06:36:44.075254  6238 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 42895a9bcba64aa484f6509605035d02 in term 14.
W20260812 06:36:44.075416  7020 leader_election.cc:336] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 222b20de5a1145c6b8a11df8374e8790 (127.4.177.5:43521): Network error: Client connection negotiation failed: client connection to 127.4.177.5:43521: connect: Connection refused (error 111)
W20260812 06:36:44.075448  6597 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.4.177.5:43521: connect: Connection refused (error 111) [suppressed 1 similar messages]
I20260812 06:36:44.075585  7020 leader_election.cc:304] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 7 responses out of 7 voters: 4 yes votes; 3 no votes. yes voters: 2df4308c852240859ada53d216ab69ae, 42895a9bcba64aa484f6509605035d02, 8d90d61da62549208fe117b4384a469b, ec6c8d818e744cdbb6b0d15b321010e0; no voters: 222b20de5a1145c6b8a11df8374e8790, 7962f587ccd842ad8a3f6ed9a5d7707e, b037d17b21f8422fa5e41b70eb4dcd08
I20260812 06:36:44.075650  7322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "42895a9bcba64aa484f6509605035d02" candidate_term: 15 candidate_status { last_received { term: 14 index: 2073 } } ignore_live_leader: false dest_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" is_pre_election: true
I20260812 06:36:44.075675  6047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "42895a9bcba64aa484f6509605035d02" candidate_term: 15 candidate_status { last_received { term: 14 index: 2073 } } ignore_live_leader: false dest_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" is_pre_election: true
I20260812 06:36:44.075803  6047 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P ec6c8d818e744cdbb6b0d15b321010e0 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 42895a9bcba64aa484f6509605035d02 in term 14.
I20260812 06:36:44.075848  9219 leader_election.cc:290] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 2df4308c852240859ada53d216ab69ae (127.4.177.6:34269), 222b20de5a1145c6b8a11df8374e8790 (127.4.177.5:43521), 7962f587ccd842ad8a3f6ed9a5d7707e (127.4.177.3:40897), 8d90d61da62549208fe117b4384a469b (127.4.177.2:41539), b037d17b21f8422fa5e41b70eb4dcd08 (127.4.177.7:46299), ec6c8d818e744cdbb6b0d15b321010e0 (127.4.177.1:41057)
I20260812 06:36:44.075942  9218 raft_consensus.cc:2804] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260812 06:36:44.076000  9218 raft_consensus.cc:493] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [term 14 FOLLOWER]: Starting leader election (detected failure of leader 222b20de5a1145c6b8a11df8374e8790)
I20260812 06:36:44.076051  9218 raft_consensus.cc:3060] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [term 14 FOLLOWER]: Advancing to term 15
I20260812 06:36:44.076164  6597 leader_election.cc:304] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 6 responses out of 7 voters: 4 yes votes; 2 no votes. yes voters: 2df4308c852240859ada53d216ab69ae, 42895a9bcba64aa484f6509605035d02, 8d90d61da62549208fe117b4384a469b, ec6c8d818e744cdbb6b0d15b321010e0; no voters: 7962f587ccd842ad8a3f6ed9a5d7707e, b037d17b21f8422fa5e41b70eb4dcd08
I20260812 06:36:44.076408  9219 raft_consensus.cc:2804] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260812 06:36:44.076484  9219 raft_consensus.cc:493] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 222b20de5a1145c6b8a11df8374e8790)
I20260812 06:36:44.076594  9219 raft_consensus.cc:3060] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [term 14 FOLLOWER]: Advancing to term 15
W20260812 06:36:44.076754  6597 leader_election.cc:336] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 222b20de5a1145c6b8a11df8374e8790 (127.4.177.5:43521): Network error: Client connection negotiation failed: client connection to 127.4.177.5:43521: connect: Connection refused (error 111)
I20260812 06:36:44.077250  9218 raft_consensus.cc:515] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df4308c852240859ada53d216ab69ae" member_type: VOTER last_known_addr { host: "127.4.177.6" port: 34269 } } peers { permanent_uuid: "42895a9bcba64aa484f6509605035d02" member_type: VOTER last_known_addr { host: "127.4.177.4" port: 38181 } } peers { permanent_uuid: "222b20de5a1145c6b8a11df8374e8790" member_type: VOTER last_known_addr { host: "127.4.177.5" port: 43521 } } peers { permanent_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 40897 } } peers { permanent_uuid: "8d90d61da62549208fe117b4384a469b" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 41539 } } peers { permanent_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" member_type: VOTER last_known_addr { host: "127.4.177.7" port: 46299 } } peers { permanent_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 41057 } }
I20260812 06:36:44.077598  9219 raft_consensus.cc:515] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df4308c852240859ada53d216ab69ae" member_type: VOTER last_known_addr { host: "127.4.177.6" port: 34269 } } peers { permanent_uuid: "42895a9bcba64aa484f6509605035d02" member_type: VOTER last_known_addr { host: "127.4.177.4" port: 38181 } } peers { permanent_uuid: "222b20de5a1145c6b8a11df8374e8790" member_type: VOTER last_known_addr { host: "127.4.177.5" port: 43521 } } peers { permanent_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 40897 } } peers { permanent_uuid: "8d90d61da62549208fe117b4384a469b" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 41539 } } peers { permanent_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" member_type: VOTER last_known_addr { host: "127.4.177.7" port: 46299 } } peers { permanent_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 41057 } }
I20260812 06:36:44.077925  9218 leader_election.cc:290] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [CANDIDATE]: Term 15 election: Requested vote from peers 42895a9bcba64aa484f6509605035d02 (127.4.177.4:38181), 222b20de5a1145c6b8a11df8374e8790 (127.4.177.5:43521), 7962f587ccd842ad8a3f6ed9a5d7707e (127.4.177.3:40897), 8d90d61da62549208fe117b4384a469b (127.4.177.2:41539), b037d17b21f8422fa5e41b70eb4dcd08 (127.4.177.7:46299), ec6c8d818e744cdbb6b0d15b321010e0 (127.4.177.1:41057)
I20260812 06:36:44.077980  6701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "2df4308c852240859ada53d216ab69ae" candidate_term: 15 candidate_status { last_received { term: 14 index: 2073 } } ignore_live_leader: false dest_uuid: "42895a9bcba64aa484f6509605035d02"
I20260812 06:36:44.078117  6701 raft_consensus.cc:2393] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 2df4308c852240859ada53d216ab69ae in current term 15: Already voted for candidate 42895a9bcba64aa484f6509605035d02 in this term.
I20260812 06:36:44.078377  8805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "2df4308c852240859ada53d216ab69ae" candidate_term: 15 candidate_status { last_received { term: 14 index: 2073 } } ignore_live_leader: false dest_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e"
I20260812 06:36:44.078655  7322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "2df4308c852240859ada53d216ab69ae" candidate_term: 15 candidate_status { last_received { term: 14 index: 2073 } } ignore_live_leader: false dest_uuid: "b037d17b21f8422fa5e41b70eb4dcd08"
I20260812 06:36:44.078753  9219 leader_election.cc:290] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [CANDIDATE]: Term 15 election: Requested vote from peers 2df4308c852240859ada53d216ab69ae (127.4.177.6:34269), 222b20de5a1145c6b8a11df8374e8790 (127.4.177.5:43521), 7962f587ccd842ad8a3f6ed9a5d7707e (127.4.177.3:40897), 8d90d61da62549208fe117b4384a469b (127.4.177.2:41539), b037d17b21f8422fa5e41b70eb4dcd08 (127.4.177.7:46299), ec6c8d818e744cdbb6b0d15b321010e0 (127.4.177.1:41057)
I20260812 06:36:44.079196  7322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "42895a9bcba64aa484f6509605035d02" candidate_term: 15 candidate_status { last_received { term: 14 index: 2073 } } ignore_live_leader: false dest_uuid: "b037d17b21f8422fa5e41b70eb4dcd08"
I20260812 06:36:44.079298  6046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "42895a9bcba64aa484f6509605035d02" candidate_term: 15 candidate_status { last_received { term: 14 index: 2073 } } ignore_live_leader: false dest_uuid: "ec6c8d818e744cdbb6b0d15b321010e0"
I20260812 06:36:44.079315  6245 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "42895a9bcba64aa484f6509605035d02" candidate_term: 15 candidate_status { last_received { term: 14 index: 2073 } } ignore_live_leader: false dest_uuid: "8d90d61da62549208fe117b4384a469b"
I20260812 06:36:44.079444  6046 raft_consensus.cc:3060] T 7065bb403e3047dd9a0853c49318ef1f P ec6c8d818e744cdbb6b0d15b321010e0 [term 14 FOLLOWER]: Advancing to term 15
I20260812 06:36:44.079444  6245 raft_consensus.cc:3060] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b [term 14 FOLLOWER]: Advancing to term 15
I20260812 06:36:44.079875  7107 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "42895a9bcba64aa484f6509605035d02" candidate_term: 15 candidate_status { last_received { term: 14 index: 2073 } } ignore_live_leader: false dest_uuid: "2df4308c852240859ada53d216ab69ae"
I20260812 06:36:44.080044  7107 raft_consensus.cc:2393] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 42895a9bcba64aa484f6509605035d02 in current term 15: Already voted for candidate 2df4308c852240859ada53d216ab69ae in this term.
I20260812 06:36:44.080469  6046 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P ec6c8d818e744cdbb6b0d15b321010e0 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 42895a9bcba64aa484f6509605035d02 in term 15.
I20260812 06:36:44.080469  6245 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 42895a9bcba64aa484f6509605035d02 in term 15.
I20260812 06:36:44.080672  8805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "42895a9bcba64aa484f6509605035d02" candidate_term: 15 candidate_status { last_received { term: 14 index: 2073 } } ignore_live_leader: false dest_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e"
I20260812 06:36:44.080821  6047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "2df4308c852240859ada53d216ab69ae" candidate_term: 15 candidate_status { last_received { term: 14 index: 2073 } } ignore_live_leader: false dest_uuid: "ec6c8d818e744cdbb6b0d15b321010e0"
I20260812 06:36:44.080857  6238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "2df4308c852240859ada53d216ab69ae" candidate_term: 15 candidate_status { last_received { term: 14 index: 2073 } } ignore_live_leader: false dest_uuid: "8d90d61da62549208fe117b4384a469b"
I20260812 06:36:44.080964  6047 raft_consensus.cc:2393] T 7065bb403e3047dd9a0853c49318ef1f P ec6c8d818e744cdbb6b0d15b321010e0 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 2df4308c852240859ada53d216ab69ae in current term 15: Already voted for candidate 42895a9bcba64aa484f6509605035d02 in this term.
I20260812 06:36:44.080967  6238 raft_consensus.cc:2393] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 2df4308c852240859ada53d216ab69ae in current term 15: Already voted for candidate 42895a9bcba64aa484f6509605035d02 in this term.
W20260812 06:36:44.081065  6597 leader_election.cc:336] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [CANDIDATE]: Term 15 election: RPC error from VoteRequest() call to peer 222b20de5a1145c6b8a11df8374e8790 (127.4.177.5:43521): Network error: Client connection negotiation failed: client connection to 127.4.177.5:43521: connect: Connection refused (error 111)
I20260812 06:36:44.081303  6597 leader_election.cc:304] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 7 responses out of 7 voters: 3 yes votes; 4 no votes. yes voters: 42895a9bcba64aa484f6509605035d02, 8d90d61da62549208fe117b4384a469b, ec6c8d818e744cdbb6b0d15b321010e0; no voters: 222b20de5a1145c6b8a11df8374e8790, 2df4308c852240859ada53d216ab69ae, 7962f587ccd842ad8a3f6ed9a5d7707e, b037d17b21f8422fa5e41b70eb4dcd08
I20260812 06:36:44.081561  9219 raft_consensus.cc:2749] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260812 06:36:44.081792  7020 leader_election.cc:304] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 5 responses out of 7 voters: 1 yes votes; 4 no votes. yes voters: 2df4308c852240859ada53d216ab69ae; no voters: 42895a9bcba64aa484f6509605035d02, 7962f587ccd842ad8a3f6ed9a5d7707e, b037d17b21f8422fa5e41b70eb4dcd08, ec6c8d818e744cdbb6b0d15b321010e0
I20260812 06:36:44.081993  9218 raft_consensus.cc:2749] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
W20260812 06:36:44.082028  7020 leader_election.cc:336] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [CANDIDATE]: Term 15 election: RPC error from VoteRequest() call to peer 222b20de5a1145c6b8a11df8374e8790 (127.4.177.5:43521): Network error: Client connection negotiation failed: client connection to 127.4.177.5:43521: connect: Connection refused (error 111)
I20260812 06:36:44.419826  9227 raft_consensus.cc:493] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 222b20de5a1145c6b8a11df8374e8790)
I20260812 06:36:44.419982  9227 raft_consensus.cc:515] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df4308c852240859ada53d216ab69ae" member_type: VOTER last_known_addr { host: "127.4.177.6" port: 34269 } } peers { permanent_uuid: "42895a9bcba64aa484f6509605035d02" member_type: VOTER last_known_addr { host: "127.4.177.4" port: 38181 } } peers { permanent_uuid: "222b20de5a1145c6b8a11df8374e8790" member_type: VOTER last_known_addr { host: "127.4.177.5" port: 43521 } } peers { permanent_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 40897 } } peers { permanent_uuid: "8d90d61da62549208fe117b4384a469b" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 41539 } } peers { permanent_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" member_type: VOTER last_known_addr { host: "127.4.177.7" port: 46299 } } peers { permanent_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 41057 } }
I20260812 06:36:44.420579  9227 leader_election.cc:290] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 2df4308c852240859ada53d216ab69ae (127.4.177.6:34269), 42895a9bcba64aa484f6509605035d02 (127.4.177.4:38181), 222b20de5a1145c6b8a11df8374e8790 (127.4.177.5:43521), 7962f587ccd842ad8a3f6ed9a5d7707e (127.4.177.3:40897), 8d90d61da62549208fe117b4384a469b (127.4.177.2:41539), ec6c8d818e744cdbb6b0d15b321010e0 (127.4.177.1:41057)
I20260812 06:36:44.421046  7107 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" candidate_term: 15 candidate_status { last_received { term: 14 index: 1984 } } ignore_live_leader: false dest_uuid: "2df4308c852240859ada53d216ab69ae" is_pre_election: true
I20260812 06:36:44.421084  6701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" candidate_term: 15 candidate_status { last_received { term: 14 index: 1984 } } ignore_live_leader: false dest_uuid: "42895a9bcba64aa484f6509605035d02" is_pre_election: true
I20260812 06:36:44.421236  6701 raft_consensus.cc:2393] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b037d17b21f8422fa5e41b70eb4dcd08 in current term 15: Already voted for candidate 42895a9bcba64aa484f6509605035d02 in this term.
I20260812 06:36:44.421234  7107 raft_consensus.cc:2393] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b037d17b21f8422fa5e41b70eb4dcd08 in current term 15: Already voted for candidate 2df4308c852240859ada53d216ab69ae in this term.
W20260812 06:36:44.421367  7211 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.4.177.5:43521: connect: Connection refused (error 111) [suppressed 1 similar messages]
I20260812 06:36:44.421799  6238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" candidate_term: 15 candidate_status { last_received { term: 14 index: 1984 } } ignore_live_leader: false dest_uuid: "8d90d61da62549208fe117b4384a469b" is_pre_election: true
I20260812 06:36:44.421815  8805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" candidate_term: 15 candidate_status { last_received { term: 14 index: 1984 } } ignore_live_leader: false dest_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" is_pre_election: true
I20260812 06:36:44.421870  6047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" candidate_term: 15 candidate_status { last_received { term: 14 index: 1984 } } ignore_live_leader: false dest_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" is_pre_election: true
I20260812 06:36:44.421975  6238 raft_consensus.cc:2393] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b037d17b21f8422fa5e41b70eb4dcd08 in current term 15: Already voted for candidate 42895a9bcba64aa484f6509605035d02 in this term.
I20260812 06:36:44.422016  6047 raft_consensus.cc:2393] T 7065bb403e3047dd9a0853c49318ef1f P ec6c8d818e744cdbb6b0d15b321010e0 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b037d17b21f8422fa5e41b70eb4dcd08 in current term 15: Already voted for candidate 42895a9bcba64aa484f6509605035d02 in this term.
I20260812 06:36:44.422403  7211 leader_election.cc:304] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 5 responses out of 7 voters: 1 yes votes; 4 no votes. yes voters: b037d17b21f8422fa5e41b70eb4dcd08; no voters: 2df4308c852240859ada53d216ab69ae, 42895a9bcba64aa484f6509605035d02, 7962f587ccd842ad8a3f6ed9a5d7707e, 8d90d61da62549208fe117b4384a469b
I20260812 06:36:44.422614  9227 raft_consensus.cc:3060] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08 [term 14 FOLLOWER]: Advancing to term 15
W20260812 06:36:44.422617  7211 leader_election.cc:336] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 222b20de5a1145c6b8a11df8374e8790 (127.4.177.5:43521): Network error: Client connection negotiation failed: client connection to 127.4.177.5:43521: connect: Connection refused (error 111)
I20260812 06:36:44.423619  9227 raft_consensus.cc:2749] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260812 06:36:44.439558  9231 raft_consensus.cc:493] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 222b20de5a1145c6b8a11df8374e8790)
I20260812 06:36:44.439719  9231 raft_consensus.cc:515] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df4308c852240859ada53d216ab69ae" member_type: VOTER last_known_addr { host: "127.4.177.6" port: 34269 } } peers { permanent_uuid: "42895a9bcba64aa484f6509605035d02" member_type: VOTER last_known_addr { host: "127.4.177.4" port: 38181 } } peers { permanent_uuid: "222b20de5a1145c6b8a11df8374e8790" member_type: VOTER last_known_addr { host: "127.4.177.5" port: 43521 } } peers { permanent_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 40897 } } peers { permanent_uuid: "8d90d61da62549208fe117b4384a469b" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 41539 } } peers { permanent_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" member_type: VOTER last_known_addr { host: "127.4.177.7" port: 46299 } } peers { permanent_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 41057 } }
I20260812 06:36:44.441195  9231 leader_election.cc:290] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 2df4308c852240859ada53d216ab69ae (127.4.177.6:34269), 42895a9bcba64aa484f6509605035d02 (127.4.177.4:38181), 222b20de5a1145c6b8a11df8374e8790 (127.4.177.5:43521), 8d90d61da62549208fe117b4384a469b (127.4.177.2:41539), b037d17b21f8422fa5e41b70eb4dcd08 (127.4.177.7:46299), ec6c8d818e744cdbb6b0d15b321010e0 (127.4.177.1:41057)
W20260812 06:36:44.442945  8698 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.4.177.5:43521: connect: Connection refused (error 111)
W20260812 06:36:44.444682  8698 leader_election.cc:336] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 222b20de5a1145c6b8a11df8374e8790 (127.4.177.5:43521): Network error: Client connection negotiation failed: client connection to 127.4.177.5:43521: connect: Connection refused (error 111)
I20260812 06:36:44.446164  6701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" candidate_term: 15 candidate_status { last_received { term: 14 index: 2069 } } ignore_live_leader: false dest_uuid: "42895a9bcba64aa484f6509605035d02" is_pre_election: true
I20260812 06:36:44.446367  6701 raft_consensus.cc:2393] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7962f587ccd842ad8a3f6ed9a5d7707e in current term 15: Already voted for candidate 42895a9bcba64aa484f6509605035d02 in this term.
I20260812 06:36:44.447295  7107 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" candidate_term: 15 candidate_status { last_received { term: 14 index: 2069 } } ignore_live_leader: false dest_uuid: "2df4308c852240859ada53d216ab69ae" is_pre_election: true
I20260812 06:36:44.447458  7107 raft_consensus.cc:2393] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7962f587ccd842ad8a3f6ed9a5d7707e in current term 15: Already voted for candidate 2df4308c852240859ada53d216ab69ae in this term.
I20260812 06:36:44.448308  6238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" candidate_term: 15 candidate_status { last_received { term: 14 index: 2069 } } ignore_live_leader: false dest_uuid: "8d90d61da62549208fe117b4384a469b" is_pre_election: true
I20260812 06:36:44.448505  6238 raft_consensus.cc:2393] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7962f587ccd842ad8a3f6ed9a5d7707e in current term 15: Already voted for candidate 42895a9bcba64aa484f6509605035d02 in this term.
I20260812 06:36:44.448658  7322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" candidate_term: 15 candidate_status { last_received { term: 14 index: 2069 } } ignore_live_leader: false dest_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" is_pre_election: true
I20260812 06:36:44.448796  7322 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7962f587ccd842ad8a3f6ed9a5d7707e in term 15.
I20260812 06:36:44.448921  8698 leader_election.cc:304] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 5 responses out of 7 voters: 1 yes votes; 4 no votes. yes voters: 7962f587ccd842ad8a3f6ed9a5d7707e; no voters: 222b20de5a1145c6b8a11df8374e8790, 2df4308c852240859ada53d216ab69ae, 42895a9bcba64aa484f6509605035d02, 8d90d61da62549208fe117b4384a469b
I20260812 06:36:44.449209  9231 raft_consensus.cc:3060] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [term 14 FOLLOWER]: Advancing to term 15
I20260812 06:36:44.449734  6047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" candidate_term: 15 candidate_status { last_received { term: 14 index: 2069 } } ignore_live_leader: false dest_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" is_pre_election: true
I20260812 06:36:44.449903  6047 raft_consensus.cc:2393] T 7065bb403e3047dd9a0853c49318ef1f P ec6c8d818e744cdbb6b0d15b321010e0 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7962f587ccd842ad8a3f6ed9a5d7707e in current term 15: Already voted for candidate 42895a9bcba64aa484f6509605035d02 in this term.
I20260812 06:36:44.450484  9231 raft_consensus.cc:2749] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260812 06:36:45.686905  9266 raft_consensus.cc:493] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 06:36:45.687069  9266 raft_consensus.cc:515] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df4308c852240859ada53d216ab69ae" member_type: VOTER last_known_addr { host: "127.4.177.6" port: 34269 } } peers { permanent_uuid: "42895a9bcba64aa484f6509605035d02" member_type: VOTER last_known_addr { host: "127.4.177.4" port: 38181 } } peers { permanent_uuid: "222b20de5a1145c6b8a11df8374e8790" member_type: VOTER last_known_addr { host: "127.4.177.5" port: 43521 } } peers { permanent_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 40897 } } peers { permanent_uuid: "8d90d61da62549208fe117b4384a469b" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 41539 } } peers { permanent_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" member_type: VOTER last_known_addr { host: "127.4.177.7" port: 46299 } } peers { permanent_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 41057 } }
I20260812 06:36:45.687882  9266 leader_election.cc:290] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 42895a9bcba64aa484f6509605035d02 (127.4.177.4:38181), 222b20de5a1145c6b8a11df8374e8790 (127.4.177.5:43521), 7962f587ccd842ad8a3f6ed9a5d7707e (127.4.177.3:40897), 8d90d61da62549208fe117b4384a469b (127.4.177.2:41539), b037d17b21f8422fa5e41b70eb4dcd08 (127.4.177.7:46299), ec6c8d818e744cdbb6b0d15b321010e0 (127.4.177.1:41057)
I20260812 06:36:45.688333  6701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "2df4308c852240859ada53d216ab69ae" candidate_term: 16 candidate_status { last_received { term: 14 index: 2073 } } ignore_live_leader: false dest_uuid: "42895a9bcba64aa484f6509605035d02" is_pre_election: true
I20260812 06:36:45.688341  7322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "2df4308c852240859ada53d216ab69ae" candidate_term: 16 candidate_status { last_received { term: 14 index: 2073 } } ignore_live_leader: false dest_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" is_pre_election: true
I20260812 06:36:45.688509  7322 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2df4308c852240859ada53d216ab69ae in term 15.
I20260812 06:36:45.688509  6701 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2df4308c852240859ada53d216ab69ae in term 15.
I20260812 06:36:45.688799  6238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "2df4308c852240859ada53d216ab69ae" candidate_term: 16 candidate_status { last_received { term: 14 index: 2073 } } ignore_live_leader: false dest_uuid: "8d90d61da62549208fe117b4384a469b" is_pre_election: true
I20260812 06:36:45.688879  8805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "2df4308c852240859ada53d216ab69ae" candidate_term: 16 candidate_status { last_received { term: 14 index: 2073 } } ignore_live_leader: false dest_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" is_pre_election: true
I20260812 06:36:45.688943  6238 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2df4308c852240859ada53d216ab69ae in term 15.
I20260812 06:36:45.689093  8805 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2df4308c852240859ada53d216ab69ae in term 15.
I20260812 06:36:45.689204  6047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "2df4308c852240859ada53d216ab69ae" candidate_term: 16 candidate_status { last_received { term: 14 index: 2073 } } ignore_live_leader: false dest_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" is_pre_election: true
I20260812 06:36:45.689350  6047 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P ec6c8d818e744cdbb6b0d15b321010e0 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2df4308c852240859ada53d216ab69ae in term 15.
I20260812 06:36:45.689409  7020 leader_election.cc:304] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 7 voters: 4 yes votes; 0 no votes. yes voters: 2df4308c852240859ada53d216ab69ae, 42895a9bcba64aa484f6509605035d02, 7962f587ccd842ad8a3f6ed9a5d7707e, b037d17b21f8422fa5e41b70eb4dcd08; no voters: 
I20260812 06:36:45.689585  9266 raft_consensus.cc:2804] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260812 06:36:45.689677  9266 raft_consensus.cc:493] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 06:36:45.689733  9266 raft_consensus.cc:3060] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [term 15 FOLLOWER]: Advancing to term 16
W20260812 06:36:45.690040  7020 leader_election.cc:336] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 222b20de5a1145c6b8a11df8374e8790 (127.4.177.5:43521): Network error: Client connection negotiation failed: client connection to 127.4.177.5:43521: connect: Connection refused (error 111)
I20260812 06:36:45.690834  9266 raft_consensus.cc:515] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df4308c852240859ada53d216ab69ae" member_type: VOTER last_known_addr { host: "127.4.177.6" port: 34269 } } peers { permanent_uuid: "42895a9bcba64aa484f6509605035d02" member_type: VOTER last_known_addr { host: "127.4.177.4" port: 38181 } } peers { permanent_uuid: "222b20de5a1145c6b8a11df8374e8790" member_type: VOTER last_known_addr { host: "127.4.177.5" port: 43521 } } peers { permanent_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 40897 } } peers { permanent_uuid: "8d90d61da62549208fe117b4384a469b" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 41539 } } peers { permanent_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" member_type: VOTER last_known_addr { host: "127.4.177.7" port: 46299 } } peers { permanent_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 41057 } }
I20260812 06:36:45.691398  9266 leader_election.cc:290] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [CANDIDATE]: Term 16 election: Requested vote from peers 42895a9bcba64aa484f6509605035d02 (127.4.177.4:38181), 222b20de5a1145c6b8a11df8374e8790 (127.4.177.5:43521), 7962f587ccd842ad8a3f6ed9a5d7707e (127.4.177.3:40897), 8d90d61da62549208fe117b4384a469b (127.4.177.2:41539), b037d17b21f8422fa5e41b70eb4dcd08 (127.4.177.7:46299), ec6c8d818e744cdbb6b0d15b321010e0 (127.4.177.1:41057)
I20260812 06:36:45.691746  7322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "2df4308c852240859ada53d216ab69ae" candidate_term: 16 candidate_status { last_received { term: 14 index: 2073 } } ignore_live_leader: false dest_uuid: "b037d17b21f8422fa5e41b70eb4dcd08"
I20260812 06:36:45.691740  6701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "2df4308c852240859ada53d216ab69ae" candidate_term: 16 candidate_status { last_received { term: 14 index: 2073 } } ignore_live_leader: false dest_uuid: "42895a9bcba64aa484f6509605035d02"
I20260812 06:36:45.691864  7322 raft_consensus.cc:3060] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08 [term 15 FOLLOWER]: Advancing to term 16
I20260812 06:36:45.691864  6701 raft_consensus.cc:3060] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [term 15 FOLLOWER]: Advancing to term 16
I20260812 06:36:45.691936  8805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "2df4308c852240859ada53d216ab69ae" candidate_term: 16 candidate_status { last_received { term: 14 index: 2073 } } ignore_live_leader: false dest_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e"
I20260812 06:36:45.692065  8805 raft_consensus.cc:3060] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [term 15 FOLLOWER]: Advancing to term 16
W20260812 06:36:45.692667  7020 leader_election.cc:336] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [CANDIDATE]: Term 16 election: RPC error from VoteRequest() call to peer 222b20de5a1145c6b8a11df8374e8790 (127.4.177.5:43521): Network error: Client connection negotiation failed: client connection to 127.4.177.5:43521: connect: Connection refused (error 111)
I20260812 06:36:45.692873  7322 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2df4308c852240859ada53d216ab69ae in term 16.
I20260812 06:36:45.692873  6701 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2df4308c852240859ada53d216ab69ae in term 16.
I20260812 06:36:45.693094  6238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "2df4308c852240859ada53d216ab69ae" candidate_term: 16 candidate_status { last_received { term: 14 index: 2073 } } ignore_live_leader: false dest_uuid: "8d90d61da62549208fe117b4384a469b"
I20260812 06:36:45.693231  6238 raft_consensus.cc:3060] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b [term 15 FOLLOWER]: Advancing to term 16
I20260812 06:36:45.693243  8805 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2df4308c852240859ada53d216ab69ae in term 16.
I20260812 06:36:45.693456  6047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "2df4308c852240859ada53d216ab69ae" candidate_term: 16 candidate_status { last_received { term: 14 index: 2073 } } ignore_live_leader: false dest_uuid: "ec6c8d818e744cdbb6b0d15b321010e0"
I20260812 06:36:45.693598  6047 raft_consensus.cc:3060] T 7065bb403e3047dd9a0853c49318ef1f P ec6c8d818e744cdbb6b0d15b321010e0 [term 15 FOLLOWER]: Advancing to term 16
I20260812 06:36:45.693698  7020 leader_election.cc:304] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 5 responses out of 7 voters: 4 yes votes; 1 no votes. yes voters: 2df4308c852240859ada53d216ab69ae, 42895a9bcba64aa484f6509605035d02, 7962f587ccd842ad8a3f6ed9a5d7707e, b037d17b21f8422fa5e41b70eb4dcd08; no voters: 222b20de5a1145c6b8a11df8374e8790
I20260812 06:36:45.693873  9266 raft_consensus.cc:2804] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [term 16 FOLLOWER]: Leader election won for term 16
I20260812 06:36:45.694036  9266 raft_consensus.cc:697] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [term 16 LEADER]: Becoming Leader. State: Replica: 2df4308c852240859ada53d216ab69ae, State: Running, Role: LEADER
I20260812 06:36:45.694208  9266 consensus_queue.cc:237] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2065, Committed index: 2065, Last appended: 14.2073, Last appended by leader: 2073, Current term: 16, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df4308c852240859ada53d216ab69ae" member_type: VOTER last_known_addr { host: "127.4.177.6" port: 34269 } } peers { permanent_uuid: "42895a9bcba64aa484f6509605035d02" member_type: VOTER last_known_addr { host: "127.4.177.4" port: 38181 } } peers { permanent_uuid: "222b20de5a1145c6b8a11df8374e8790" member_type: VOTER last_known_addr { host: "127.4.177.5" port: 43521 } } peers { permanent_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 40897 } } peers { permanent_uuid: "8d90d61da62549208fe117b4384a469b" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 41539 } } peers { permanent_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" member_type: VOTER last_known_addr { host: "127.4.177.7" port: 46299 } } peers { permanent_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 41057 } }
I20260812 06:36:45.694458  6238 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2df4308c852240859ada53d216ab69ae in term 16.
I20260812 06:36:45.694783  6047 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P ec6c8d818e744cdbb6b0d15b321010e0 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2df4308c852240859ada53d216ab69ae in term 16.
I20260812 06:36:45.696823  5812 catalog_manager.cc:5719] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae reported cstate change: term changed from 14 to 16, leader changed from 222b20de5a1145c6b8a11df8374e8790 (127.4.177.5) to 2df4308c852240859ada53d216ab69ae (127.4.177.6). New cstate: current_term: 16 leader_uuid: "2df4308c852240859ada53d216ab69ae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df4308c852240859ada53d216ab69ae" member_type: VOTER last_known_addr { host: "127.4.177.6" port: 34269 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "42895a9bcba64aa484f6509605035d02" member_type: VOTER last_known_addr { host: "127.4.177.4" port: 38181 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "222b20de5a1145c6b8a11df8374e8790" member_type: VOTER last_known_addr { host: "127.4.177.5" port: 43521 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 40897 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8d90d61da62549208fe117b4384a469b" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 41539 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" member_type: VOTER last_known_addr { host: "127.4.177.7" port: 46299 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 41057 } health_report { overall_health: UNKNOWN } } }
I20260812 06:36:45.803184  4804 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskCnZu03/build/debug/bin/kudu
/tmp/dist-test-taskCnZu03/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-4/wal
--fs_data_dirs=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-4/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-4/logs
--server_dump_info_path=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-4/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.177.5:43521
--local_ip_for_outbound_sockets=127.4.177.5
--tserver_master_addrs=127.4.177.62:42183
--webserver_port=43111
--webserver_interface=127.4.177.5
--builtin_ntp_servers=127.4.177.20:34335
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_cache_size_limit_mb=10
--consensus_rpc_timeout_ms=1000 with env {}
W20260812 06:36:45.915136  9284 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:36:45.915349  9284 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:36:45.915403  9284 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 06:36:45.919502  9284 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:36:45.919653  9284 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.177.5
I20260812 06:36:45.925451  9284 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-4/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.177.20:34335
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--consensus_rpc_timeout_ms=1000
--log_cache_size_limit_mb=10
--fs_data_dirs=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-4/data
--fs_wal_dir=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-4/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.177.5:43521
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-4/data/info.pb
--webserver_interface=127.4.177.5
--webserver_port=43111
--tserver_master_addrs=127.4.177.62:42183
--never_fsync=true
--heap_profile_path=/tmp/kudu.9284
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.177.5

Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 06:36:45.927069  9284 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:36:45.928267  9284 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 06:36:45.935897  9299 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:36:45.935854  9291 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:36:45.935854  9294 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:36:45.935992  9284 server_base.cc:1061] running on GCE node
I20260812 06:36:45.936502  9284 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:36:45.937101  9284 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:36:45.938311  9284 hybrid_clock.cc:648] HybridClock initialized: now 1786516605938289 us; error 35 us; skew 500 ppm
I20260812 06:36:45.940117  9284 webserver.cc:533] Webserver started at http://127.4.177.5:43111/ using document root <none> and password file <none>
I20260812 06:36:45.940704  9284 fs_manager.cc:362] Metadata directory not provided
I20260812 06:36:45.940761  9284 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:36:45.944259  9284 fs_manager.cc:714] Time spent opening directory manager: real 0.002s	user 0.003s	sys 0.000s
I20260812 06:36:45.946182  9308 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 06:36:45.947299  9284 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260812 06:36:45.947453  9284 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-4/data,/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-4/wal
uuid: "222b20de5a1145c6b8a11df8374e8790"
format_stamp: "Formatted at 2026-08-12 06:36:11 on dist-test-slave-jztv"
I20260812 06:36:45.947871  9284 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-4/wal
metadata directory: /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-4/wal
1 data directories: /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-4/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:36:45.968650  9284 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 06:36:45.969353  9284 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:36:45.969565  9284 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:36:45.970264  9284 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260812 06:36:45.971529  9318 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
I20260812 06:36:45.974852  9284 ts_tablet_manager.cc:585] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260812 06:36:45.974956  9284 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.004s	user 0.000s	sys 0.000s
I20260812 06:36:45.975056  9284 ts_tablet_manager.cc:600] Registering tablets (0/1 complete)
I20260812 06:36:45.976596  9284 ts_tablet_manager.cc:616] Registered 1 tablets
I20260812 06:36:45.976670  9284 ts_tablet_manager.cc:595] Time spent register tablets: real 0.002s	user 0.000s	sys 0.000s
I20260812 06:36:45.976986  9318 tablet_bootstrap.cc:492] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790: Bootstrap starting.
I20260812 06:36:45.988082  9284 rpc_server.cc:307] RPC server started. Bound to: 127.4.177.5:43521
I20260812 06:36:45.988152  9493 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.4.177.5:43521 every 8 connection(s)
I20260812 06:36:45.989097  9284 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-4/data/info.pb
I20260812 06:36:45.991318  4804 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskCnZu03/build/debug/bin/kudu as pid 9284
I20260812 06:36:45.999406  9494 heartbeater.cc:344] Connected to a master server at 127.4.177.62:42183
I20260812 06:36:45.999688  9494 heartbeater.cc:461] Registering TS with master...
I20260812 06:36:46.000203  9494 heartbeater.cc:507] Master 127.4.177.62:42183 requested a full tablet report, sending...
I20260812 06:36:46.001919  5812 ts_manager.cc:194] Re-registered known tserver with Master: 222b20de5a1145c6b8a11df8374e8790 (127.4.177.5:43521)
I20260812 06:36:46.002907  5812 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.4.177.5:46269
I20260812 06:36:46.111145  9318 log.cc:826] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790: Log is configured to *not* fsync() on all Append() calls
I20260812 06:36:46.162088  6238 raft_consensus.cc:1275] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b [term 16 FOLLOWER]: Refusing update from remote peer 2df4308c852240859ada53d216ab69ae: Log matching property violated. Preceding OpId in replica: term: 14 index: 2019. Preceding OpId from leader: term: 16 index: 2074. (index mismatch)
I20260812 06:36:46.162703  9266 consensus_queue.cc:1048] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "8d90d61da62549208fe117b4384a469b" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 41539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2074, Last known committed idx: 2019, Time since last communication: 0.000s
I20260812 06:36:46.179078  8805 raft_consensus.cc:1275] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [term 16 FOLLOWER]: Refusing update from remote peer 2df4308c852240859ada53d216ab69ae: Log matching property violated. Preceding OpId in replica: term: 14 index: 2069. Preceding OpId from leader: term: 16 index: 2074. (index mismatch)
I20260812 06:36:46.179672  9266 consensus_queue.cc:1048] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 40897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2074, Last known committed idx: 2065, Time since last communication: 0.000s
W20260812 06:36:46.194381  7020 consensus_peers.cc:597] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae -> Peer 222b20de5a1145c6b8a11df8374e8790 (127.4.177.5:43521): Couldn't send request to peer 222b20de5a1145c6b8a11df8374e8790. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260812 06:36:46.197573  6701 raft_consensus.cc:1275] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [term 16 FOLLOWER]: Refusing update from remote peer 2df4308c852240859ada53d216ab69ae: Log matching property violated. Preceding OpId in replica: term: 14 index: 2073. Preceding OpId from leader: term: 16 index: 2074. (index mismatch)
I20260812 06:36:46.198151  9266 consensus_queue.cc:1048] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "42895a9bcba64aa484f6509605035d02" member_type: VOTER last_known_addr { host: "127.4.177.4" port: 38181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2074, Last known committed idx: 2065, Time since last communication: 0.000s
I20260812 06:36:46.202324  7322 raft_consensus.cc:1275] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08 [term 16 FOLLOWER]: Refusing update from remote peer 2df4308c852240859ada53d216ab69ae: Log matching property violated. Preceding OpId in replica: term: 14 index: 1984. Preceding OpId from leader: term: 16 index: 2074. (index mismatch)
I20260812 06:36:46.203317  9538 consensus_queue.cc:1048] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" member_type: VOTER last_known_addr { host: "127.4.177.7" port: 46299 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2074, Last known committed idx: 1979, Time since last communication: 0.000s
I20260812 06:36:46.207075  6047 raft_consensus.cc:1275] T 7065bb403e3047dd9a0853c49318ef1f P ec6c8d818e744cdbb6b0d15b321010e0 [term 16 FOLLOWER]: Refusing update from remote peer 2df4308c852240859ada53d216ab69ae: Log matching property violated. Preceding OpId in replica: term: 14 index: 1949. Preceding OpId from leader: term: 16 index: 2074. (index mismatch)
I20260812 06:36:46.208036  9544 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:50492 (ReqId={client: 3067f3aef2a44cc4bb7f2ccd6acf83db, seq_no=2055, attempt_no=1}) took 3281 ms. Trace:
I20260812 06:36:46.208190  9544 rpcz_store.cc:276] 0812 06:36:42.926643 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 06:36:42.927372 (+   729us) service_pool.cc:224] Handling call
0812 06:36:46.208020 (+3280648us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260812 06:36:46.208506  9266 consensus_queue.cc:1048] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 41057 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2074, Last known committed idx: 1947, Time since last communication: 0.000s
I20260812 06:36:46.210444  9544 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:50492 (ReqId={client: 3067f3aef2a44cc4bb7f2ccd6acf83db, seq_no=2056, attempt_no=1}) took 3283 ms. Trace:
I20260812 06:36:46.210565  9544 rpcz_store.cc:276] 0812 06:36:42.926692 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 06:36:42.927103 (+   411us) service_pool.cc:224] Handling call
0812 06:36:46.210429 (+3283326us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260812 06:36:46.222078  9542 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:50492 (ReqId={client: 3067f3aef2a44cc4bb7f2ccd6acf83db, seq_no=2058, attempt_no=1}) took 3295 ms. Trace:
I20260812 06:36:46.222278  9542 rpcz_store.cc:276] 0812 06:36:42.926843 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 06:36:42.927043 (+   200us) service_pool.cc:224] Handling call
0812 06:36:46.222059 (+3295016us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260812 06:36:46.223233  9542 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:50492 (ReqId={client: 3067f3aef2a44cc4bb7f2ccd6acf83db, seq_no=2060, attempt_no=1}) took 3296 ms. Trace:
I20260812 06:36:46.223311  9542 rpcz_store.cc:276] 0812 06:36:42.926901 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 06:36:42.926929 (+    28us) service_pool.cc:224] Handling call
0812 06:36:46.223220 (+3296291us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260812 06:36:46.225193  9542 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:50492 (ReqId={client: 3067f3aef2a44cc4bb7f2ccd6acf83db, seq_no=2061, attempt_no=1}) took 3298 ms. Trace:
I20260812 06:36:46.225267  9542 rpcz_store.cc:276] 0812 06:36:42.926732 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 06:36:42.926772 (+    40us) service_pool.cc:224] Handling call
0812 06:36:46.225181 (+3298409us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260812 06:36:46.226140  9542 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:50492 (ReqId={client: 3067f3aef2a44cc4bb7f2ccd6acf83db, seq_no=2059, attempt_no=1}) took 3299 ms. Trace:
I20260812 06:36:46.226205  9542 rpcz_store.cc:276] 0812 06:36:42.926571 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 06:36:42.926631 (+    60us) service_pool.cc:224] Handling call
0812 06:36:46.226128 (+3299497us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260812 06:36:46.231477  9542 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:50492 (ReqId={client: 3067f3aef2a44cc4bb7f2ccd6acf83db, seq_no=2062, attempt_no=1}) took 3304 ms. Trace:
I20260812 06:36:46.231547  9542 rpcz_store.cc:276] 0812 06:36:42.926787 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 06:36:42.926817 (+    30us) service_pool.cc:224] Handling call
0812 06:36:46.231463 (+3304646us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260812 06:36:46.234825  9544 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:50492 (ReqId={client: 3067f3aef2a44cc4bb7f2ccd6acf83db, seq_no=2057, attempt_no=1}) took 3307 ms. Trace:
I20260812 06:36:46.234905  9544 rpcz_store.cc:276] 0812 06:36:42.926954 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 06:36:42.926982 (+    28us) service_pool.cc:224] Handling call
0812 06:36:46.234813 (+3307831us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260812 06:36:47.005246  9494 heartbeater.cc:499] Master 127.4.177.62:42183 was elected leader, sending a full tablet report...
I20260812 06:36:47.446113  4804 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskCnZu03/build/debug/bin/kudu with pid 6990
W20260812 06:36:47.459682  7401 meta_cache.cc:302] tablet 7065bb403e3047dd9a0853c49318ef1f: replica 2df4308c852240859ada53d216ab69ae (127.4.177.6:34269) has failed: Network error: recv got EOF from 127.4.177.6:34269 (error 108)
W20260812 06:36:47.466724  7209 connection.cc:466] server connection from 127.4.177.6:46809 torn down before Call kudu.consensus.ConsensusService.UpdateConsensus from 127.4.177.6:46809 (request call id 230) could send its response
W20260812 06:36:47.469942  6148 connection.cc:466] server connection from 127.4.177.6:49219 torn down before Call kudu.consensus.ConsensusService.UpdateConsensus from 127.4.177.6:49219 (request call id 231) could send its response
W20260812 06:36:47.470155  5939 connection.cc:466] server connection from 127.4.177.6:57961 torn down before Call kudu.consensus.ConsensusService.UpdateConsensus from 127.4.177.6:57961 (request call id 125) could send its response
W20260812 06:36:47.476482  9313 connection.cc:466] server connection from 127.4.177.6:47393 torn down before Call kudu.consensus.ConsensusService.UpdateConsensus from 127.4.177.6:47393 (request call id 2) could send its response
W20260812 06:36:47.482649  6596 connection.cc:466] server connection from 127.4.177.6:39273 torn down before Call kudu.consensus.ConsensusService.UpdateConsensus from 127.4.177.6:39273 (request call id 253) could send its response
I20260812 06:36:48.034864  9318 tablet_bootstrap.cc:492] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790: Bootstrap replayed 1/1 log segments. Stats: ops{read=2083 overwritten=10 applied=2065 ignored=0} inserts{seen=20550 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 8 replicates
I20260812 06:36:48.035454  9318 tablet_bootstrap.cc:492] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790: Bootstrap complete.
I20260812 06:36:48.037153  9318 ts_tablet_manager.cc:1403] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790: Time spent bootstrapping tablet: real 2.060s	user 1.685s	sys 0.071s
I20260812 06:36:48.039224  9318 raft_consensus.cc:359] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [term 14 FOLLOWER]: Replica starting. Triggering 8 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df4308c852240859ada53d216ab69ae" member_type: VOTER last_known_addr { host: "127.4.177.6" port: 34269 } } peers { permanent_uuid: "42895a9bcba64aa484f6509605035d02" member_type: VOTER last_known_addr { host: "127.4.177.4" port: 38181 } } peers { permanent_uuid: "222b20de5a1145c6b8a11df8374e8790" member_type: VOTER last_known_addr { host: "127.4.177.5" port: 43521 } } peers { permanent_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 40897 } } peers { permanent_uuid: "8d90d61da62549208fe117b4384a469b" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 41539 } } peers { permanent_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" member_type: VOTER last_known_addr { host: "127.4.177.7" port: 46299 } } peers { permanent_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 41057 } }
I20260812 06:36:48.041045  9318 raft_consensus.cc:740] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [term 14 FOLLOWER]: Becoming Follower/Learner. State: Replica: 222b20de5a1145c6b8a11df8374e8790, State: Initialized, Role: FOLLOWER
I20260812 06:36:48.041563  9318 consensus_queue.cc:260] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2065, Last appended: 14.2073, Last appended by leader: 2073, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df4308c852240859ada53d216ab69ae" member_type: VOTER last_known_addr { host: "127.4.177.6" port: 34269 } } peers { permanent_uuid: "42895a9bcba64aa484f6509605035d02" member_type: VOTER last_known_addr { host: "127.4.177.4" port: 38181 } } peers { permanent_uuid: "222b20de5a1145c6b8a11df8374e8790" member_type: VOTER last_known_addr { host: "127.4.177.5" port: 43521 } } peers { permanent_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 40897 } } peers { permanent_uuid: "8d90d61da62549208fe117b4384a469b" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 41539 } } peers { permanent_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" member_type: VOTER last_known_addr { host: "127.4.177.7" port: 46299 } } peers { permanent_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 41057 } }
I20260812 06:36:48.042393  9318 ts_tablet_manager.cc:1434] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790: Time spent starting tablet: real 0.005s	user 0.008s	sys 0.000s
I20260812 06:36:48.527856  9617 raft_consensus.cc:493] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 2df4308c852240859ada53d216ab69ae)
I20260812 06:36:48.528035  9617 raft_consensus.cc:515] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df4308c852240859ada53d216ab69ae" member_type: VOTER last_known_addr { host: "127.4.177.6" port: 34269 } } peers { permanent_uuid: "42895a9bcba64aa484f6509605035d02" member_type: VOTER last_known_addr { host: "127.4.177.4" port: 38181 } } peers { permanent_uuid: "222b20de5a1145c6b8a11df8374e8790" member_type: VOTER last_known_addr { host: "127.4.177.5" port: 43521 } } peers { permanent_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 40897 } } peers { permanent_uuid: "8d90d61da62549208fe117b4384a469b" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 41539 } } peers { permanent_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" member_type: VOTER last_known_addr { host: "127.4.177.7" port: 46299 } } peers { permanent_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 41057 } }
I20260812 06:36:48.528762  9617 leader_election.cc:290] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 2df4308c852240859ada53d216ab69ae (127.4.177.6:34269), 42895a9bcba64aa484f6509605035d02 (127.4.177.4:38181), 222b20de5a1145c6b8a11df8374e8790 (127.4.177.5:43521), 8d90d61da62549208fe117b4384a469b (127.4.177.2:41539), b037d17b21f8422fa5e41b70eb4dcd08 (127.4.177.7:46299), ec6c8d818e744cdbb6b0d15b321010e0 (127.4.177.1:41057)
I20260812 06:36:48.529186  7322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" candidate_term: 17 candidate_status { last_received { term: 16 index: 2228 } } ignore_live_leader: false dest_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" is_pre_election: true
I20260812 06:36:48.529551  6692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" candidate_term: 17 candidate_status { last_received { term: 16 index: 2228 } } ignore_live_leader: false dest_uuid: "42895a9bcba64aa484f6509605035d02" is_pre_election: true
I20260812 06:36:48.529844  6046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" candidate_term: 17 candidate_status { last_received { term: 16 index: 2228 } } ignore_live_leader: false dest_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" is_pre_election: true
I20260812 06:36:48.529959  6245 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" candidate_term: 17 candidate_status { last_received { term: 16 index: 2228 } } ignore_live_leader: false dest_uuid: "8d90d61da62549208fe117b4384a469b" is_pre_election: true
I20260812 06:36:48.530380  8698 leader_election.cc:304] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 5 responses out of 7 voters: 1 yes votes; 4 no votes. yes voters: 7962f587ccd842ad8a3f6ed9a5d7707e; no voters: 42895a9bcba64aa484f6509605035d02, 8d90d61da62549208fe117b4384a469b, b037d17b21f8422fa5e41b70eb4dcd08, ec6c8d818e744cdbb6b0d15b321010e0
I20260812 06:36:48.530566  9617 raft_consensus.cc:2749] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260812 06:36:48.531039  8698 leader_election.cc:336] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 2df4308c852240859ada53d216ab69ae (127.4.177.6:34269): Network error: Client connection negotiation failed: client connection to 127.4.177.6:34269: connect: Connection refused (error 111)
I20260812 06:36:48.534322  9425 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" candidate_term: 17 candidate_status { last_received { term: 16 index: 2228 } } ignore_live_leader: false dest_uuid: "222b20de5a1145c6b8a11df8374e8790" is_pre_election: true
I20260812 06:36:48.534670  9425 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7962f587ccd842ad8a3f6ed9a5d7707e in term 14.
I20260812 06:36:48.963697  9629 raft_consensus.cc:493] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 2df4308c852240859ada53d216ab69ae)
I20260812 06:36:48.963876  9629 raft_consensus.cc:515] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df4308c852240859ada53d216ab69ae" member_type: VOTER last_known_addr { host: "127.4.177.6" port: 34269 } } peers { permanent_uuid: "42895a9bcba64aa484f6509605035d02" member_type: VOTER last_known_addr { host: "127.4.177.4" port: 38181 } } peers { permanent_uuid: "222b20de5a1145c6b8a11df8374e8790" member_type: VOTER last_known_addr { host: "127.4.177.5" port: 43521 } } peers { permanent_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 40897 } } peers { permanent_uuid: "8d90d61da62549208fe117b4384a469b" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 41539 } } peers { permanent_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" member_type: VOTER last_known_addr { host: "127.4.177.7" port: 46299 } } peers { permanent_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 41057 } }
I20260812 06:36:48.964411  9630 raft_consensus.cc:493] T 7065bb403e3047dd9a0853c49318ef1f P ec6c8d818e744cdbb6b0d15b321010e0 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 2df4308c852240859ada53d216ab69ae)
I20260812 06:36:48.964524  9630 raft_consensus.cc:515] T 7065bb403e3047dd9a0853c49318ef1f P ec6c8d818e744cdbb6b0d15b321010e0 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df4308c852240859ada53d216ab69ae" member_type: VOTER last_known_addr { host: "127.4.177.6" port: 34269 } } peers { permanent_uuid: "42895a9bcba64aa484f6509605035d02" member_type: VOTER last_known_addr { host: "127.4.177.4" port: 38181 } } peers { permanent_uuid: "222b20de5a1145c6b8a11df8374e8790" member_type: VOTER last_known_addr { host: "127.4.177.5" port: 43521 } } peers { permanent_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 40897 } } peers { permanent_uuid: "8d90d61da62549208fe117b4384a469b" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 41539 } } peers { permanent_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" member_type: VOTER last_known_addr { host: "127.4.177.7" port: 46299 } } peers { permanent_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 41057 } }
I20260812 06:36:48.964882  9629 leader_election.cc:290] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 2df4308c852240859ada53d216ab69ae (127.4.177.6:34269), 42895a9bcba64aa484f6509605035d02 (127.4.177.4:38181), 222b20de5a1145c6b8a11df8374e8790 (127.4.177.5:43521), 7962f587ccd842ad8a3f6ed9a5d7707e (127.4.177.3:40897), 8d90d61da62549208fe117b4384a469b (127.4.177.2:41539), ec6c8d818e744cdbb6b0d15b321010e0 (127.4.177.1:41057)
I20260812 06:36:48.965528  9630 leader_election.cc:290] T 7065bb403e3047dd9a0853c49318ef1f P ec6c8d818e744cdbb6b0d15b321010e0 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 2df4308c852240859ada53d216ab69ae (127.4.177.6:34269), 42895a9bcba64aa484f6509605035d02 (127.4.177.4:38181), 222b20de5a1145c6b8a11df8374e8790 (127.4.177.5:43521), 7962f587ccd842ad8a3f6ed9a5d7707e (127.4.177.3:40897), 8d90d61da62549208fe117b4384a469b (127.4.177.2:41539), b037d17b21f8422fa5e41b70eb4dcd08 (127.4.177.7:46299)
I20260812 06:36:48.966320  6692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" candidate_term: 17 candidate_status { last_received { term: 16 index: 2191 } } ignore_live_leader: false dest_uuid: "42895a9bcba64aa484f6509605035d02" is_pre_election: true
I20260812 06:36:48.966398  8804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" candidate_term: 17 candidate_status { last_received { term: 16 index: 2191 } } ignore_live_leader: false dest_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" is_pre_election: true
I20260812 06:36:48.966444  6701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" candidate_term: 17 candidate_status { last_received { term: 16 index: 2161 } } ignore_live_leader: false dest_uuid: "42895a9bcba64aa484f6509605035d02" is_pre_election: true
I20260812 06:36:48.966591  8804 raft_consensus.cc:2410] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b037d17b21f8422fa5e41b70eb4dcd08 for term 17 because replica has last-logged OpId of term: 16 index: 2228, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 2191.
I20260812 06:36:48.966960  6238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" candidate_term: 17 candidate_status { last_received { term: 16 index: 2161 } } ignore_live_leader: false dest_uuid: "8d90d61da62549208fe117b4384a469b" is_pre_election: true
W20260812 06:36:48.967083  7211 leader_election.cc:336] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 2df4308c852240859ada53d216ab69ae (127.4.177.6:34269): Network error: Client connection negotiation failed: client connection to 127.4.177.6:34269: connect: Connection refused (error 111)
I20260812 06:36:48.967257  6245 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" candidate_term: 17 candidate_status { last_received { term: 16 index: 2191 } } ignore_live_leader: false dest_uuid: "8d90d61da62549208fe117b4384a469b" is_pre_election: true
I20260812 06:36:48.967443  8805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" candidate_term: 17 candidate_status { last_received { term: 16 index: 2161 } } ignore_live_leader: false dest_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" is_pre_election: true
I20260812 06:36:48.967617  8805 raft_consensus.cc:2410] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ec6c8d818e744cdbb6b0d15b321010e0 for term 17 because replica has last-logged OpId of term: 16 index: 2228, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 2161.
I20260812 06:36:48.967715  7211 leader_election.cc:304] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 5 responses out of 7 voters: 1 yes votes; 4 no votes. yes voters: b037d17b21f8422fa5e41b70eb4dcd08; no voters: 2df4308c852240859ada53d216ab69ae, 42895a9bcba64aa484f6509605035d02, 7962f587ccd842ad8a3f6ed9a5d7707e, 8d90d61da62549208fe117b4384a469b
W20260812 06:36:48.967777  5945 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.4.177.6:34269: connect: Connection refused (error 111)
I20260812 06:36:48.967916  9629 raft_consensus.cc:2749] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260812 06:36:48.968075  6046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" candidate_term: 17 candidate_status { last_received { term: 16 index: 2191 } } ignore_live_leader: false dest_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" is_pre_election: true
I20260812 06:36:48.968075  7322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" candidate_term: 17 candidate_status { last_received { term: 16 index: 2161 } } ignore_live_leader: false dest_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" is_pre_election: true
I20260812 06:36:48.968195  6046 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P ec6c8d818e744cdbb6b0d15b321010e0 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b037d17b21f8422fa5e41b70eb4dcd08 in term 16.
I20260812 06:36:48.968222  7322 raft_consensus.cc:2410] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ec6c8d818e744cdbb6b0d15b321010e0 for term 17 because replica has last-logged OpId of term: 16 index: 2191, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 2161.
I20260812 06:36:48.968497  5940 leader_election.cc:304] T 7065bb403e3047dd9a0853c49318ef1f P ec6c8d818e744cdbb6b0d15b321010e0 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 5 responses out of 7 voters: 1 yes votes; 4 no votes. yes voters: ec6c8d818e744cdbb6b0d15b321010e0; no voters: 42895a9bcba64aa484f6509605035d02, 7962f587ccd842ad8a3f6ed9a5d7707e, 8d90d61da62549208fe117b4384a469b, b037d17b21f8422fa5e41b70eb4dcd08
I20260812 06:36:48.968674  9630 raft_consensus.cc:2749] T 7065bb403e3047dd9a0853c49318ef1f P ec6c8d818e744cdbb6b0d15b321010e0 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260812 06:36:48.969035  5945 leader_election.cc:336] T 7065bb403e3047dd9a0853c49318ef1f P ec6c8d818e744cdbb6b0d15b321010e0 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 2df4308c852240859ada53d216ab69ae (127.4.177.6:34269): Network error: Client connection negotiation failed: client connection to 127.4.177.6:34269: connect: Connection refused (error 111)
I20260812 06:36:48.969725  9644 raft_consensus.cc:493] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 2df4308c852240859ada53d216ab69ae)
I20260812 06:36:48.969825  9644 raft_consensus.cc:515] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df4308c852240859ada53d216ab69ae" member_type: VOTER last_known_addr { host: "127.4.177.6" port: 34269 } } peers { permanent_uuid: "42895a9bcba64aa484f6509605035d02" member_type: VOTER last_known_addr { host: "127.4.177.4" port: 38181 } } peers { permanent_uuid: "222b20de5a1145c6b8a11df8374e8790" member_type: VOTER last_known_addr { host: "127.4.177.5" port: 43521 } } peers { permanent_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 40897 } } peers { permanent_uuid: "8d90d61da62549208fe117b4384a469b" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 41539 } } peers { permanent_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" member_type: VOTER last_known_addr { host: "127.4.177.7" port: 46299 } } peers { permanent_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 41057 } }
I20260812 06:36:48.970979  7322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "8d90d61da62549208fe117b4384a469b" candidate_term: 17 candidate_status { last_received { term: 16 index: 2220 } } ignore_live_leader: false dest_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" is_pre_election: true
I20260812 06:36:48.971109  7322 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8d90d61da62549208fe117b4384a469b in term 16.
I20260812 06:36:48.971623  8805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "8d90d61da62549208fe117b4384a469b" candidate_term: 17 candidate_status { last_received { term: 16 index: 2220 } } ignore_live_leader: false dest_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" is_pre_election: true
I20260812 06:36:48.971633  6692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "8d90d61da62549208fe117b4384a469b" candidate_term: 17 candidate_status { last_received { term: 16 index: 2220 } } ignore_live_leader: false dest_uuid: "42895a9bcba64aa484f6509605035d02" is_pre_election: true
I20260812 06:36:48.971784  8805 raft_consensus.cc:2410] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8d90d61da62549208fe117b4384a469b for term 17 because replica has last-logged OpId of term: 16 index: 2228, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 2220.
I20260812 06:36:48.971809  6046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "8d90d61da62549208fe117b4384a469b" candidate_term: 17 candidate_status { last_received { term: 16 index: 2220 } } ignore_live_leader: false dest_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" is_pre_election: true
I20260812 06:36:48.971943  6046 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P ec6c8d818e744cdbb6b0d15b321010e0 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8d90d61da62549208fe117b4384a469b in term 16.
I20260812 06:36:48.972524  9425 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" candidate_term: 17 candidate_status { last_received { term: 16 index: 2161 } } ignore_live_leader: false dest_uuid: "222b20de5a1145c6b8a11df8374e8790" is_pre_election: true
I20260812 06:36:48.972669  9425 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec6c8d818e744cdbb6b0d15b321010e0 in term 14.
W20260812 06:36:48.972697  6150 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.4.177.6:34269: connect: Connection refused (error 111)
I20260812 06:36:48.973004  9423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" candidate_term: 17 candidate_status { last_received { term: 16 index: 2191 } } ignore_live_leader: false dest_uuid: "222b20de5a1145c6b8a11df8374e8790" is_pre_election: true
I20260812 06:36:48.973129  9423 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b037d17b21f8422fa5e41b70eb4dcd08 in term 14.
I20260812 06:36:48.973359  9644 leader_election.cc:290] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 2df4308c852240859ada53d216ab69ae (127.4.177.6:34269), 42895a9bcba64aa484f6509605035d02 (127.4.177.4:38181), 222b20de5a1145c6b8a11df8374e8790 (127.4.177.5:43521), 7962f587ccd842ad8a3f6ed9a5d7707e (127.4.177.3:40897), b037d17b21f8422fa5e41b70eb4dcd08 (127.4.177.7:46299), ec6c8d818e744cdbb6b0d15b321010e0 (127.4.177.1:41057)
W20260812 06:36:48.973599  6150 leader_election.cc:336] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 2df4308c852240859ada53d216ab69ae (127.4.177.6:34269): Network error: Client connection negotiation failed: client connection to 127.4.177.6:34269: connect: Connection refused (error 111)
I20260812 06:36:48.975682  9423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "8d90d61da62549208fe117b4384a469b" candidate_term: 17 candidate_status { last_received { term: 16 index: 2220 } } ignore_live_leader: false dest_uuid: "222b20de5a1145c6b8a11df8374e8790" is_pre_election: true
I20260812 06:36:48.975777  9423 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8d90d61da62549208fe117b4384a469b in term 14.
I20260812 06:36:48.976126  6150 leader_election.cc:304] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 7 responses out of 7 voters: 4 yes votes; 3 no votes. yes voters: 222b20de5a1145c6b8a11df8374e8790, 8d90d61da62549208fe117b4384a469b, b037d17b21f8422fa5e41b70eb4dcd08, ec6c8d818e744cdbb6b0d15b321010e0; no voters: 2df4308c852240859ada53d216ab69ae, 42895a9bcba64aa484f6509605035d02, 7962f587ccd842ad8a3f6ed9a5d7707e
I20260812 06:36:48.976313  9644 raft_consensus.cc:2804] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260812 06:36:48.976375  9644 raft_consensus.cc:493] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b [term 16 FOLLOWER]: Starting leader election (detected failure of leader 2df4308c852240859ada53d216ab69ae)
I20260812 06:36:48.976428  9644 raft_consensus.cc:3060] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b [term 16 FOLLOWER]: Advancing to term 17
I20260812 06:36:48.977275  9644 raft_consensus.cc:515] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df4308c852240859ada53d216ab69ae" member_type: VOTER last_known_addr { host: "127.4.177.6" port: 34269 } } peers { permanent_uuid: "42895a9bcba64aa484f6509605035d02" member_type: VOTER last_known_addr { host: "127.4.177.4" port: 38181 } } peers { permanent_uuid: "222b20de5a1145c6b8a11df8374e8790" member_type: VOTER last_known_addr { host: "127.4.177.5" port: 43521 } } peers { permanent_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 40897 } } peers { permanent_uuid: "8d90d61da62549208fe117b4384a469b" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 41539 } } peers { permanent_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" member_type: VOTER last_known_addr { host: "127.4.177.7" port: 46299 } } peers { permanent_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 41057 } }
I20260812 06:36:48.977921  9644 leader_election.cc:290] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b [CANDIDATE]: Term 17 election: Requested vote from peers 2df4308c852240859ada53d216ab69ae (127.4.177.6:34269), 42895a9bcba64aa484f6509605035d02 (127.4.177.4:38181), 222b20de5a1145c6b8a11df8374e8790 (127.4.177.5:43521), 7962f587ccd842ad8a3f6ed9a5d7707e (127.4.177.3:40897), b037d17b21f8422fa5e41b70eb4dcd08 (127.4.177.7:46299), ec6c8d818e744cdbb6b0d15b321010e0 (127.4.177.1:41057)
I20260812 06:36:48.978257  7322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "8d90d61da62549208fe117b4384a469b" candidate_term: 17 candidate_status { last_received { term: 16 index: 2220 } } ignore_live_leader: false dest_uuid: "b037d17b21f8422fa5e41b70eb4dcd08"
I20260812 06:36:48.978278  9423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "8d90d61da62549208fe117b4384a469b" candidate_term: 17 candidate_status { last_received { term: 16 index: 2220 } } ignore_live_leader: false dest_uuid: "222b20de5a1145c6b8a11df8374e8790"
I20260812 06:36:48.978374  7322 raft_consensus.cc:3060] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08 [term 16 FOLLOWER]: Advancing to term 17
I20260812 06:36:48.978384  9423 raft_consensus.cc:3060] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [term 14 FOLLOWER]: Advancing to term 17
I20260812 06:36:48.978493  6046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "8d90d61da62549208fe117b4384a469b" candidate_term: 17 candidate_status { last_received { term: 16 index: 2220 } } ignore_live_leader: false dest_uuid: "ec6c8d818e744cdbb6b0d15b321010e0"
I20260812 06:36:48.978513  8805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "8d90d61da62549208fe117b4384a469b" candidate_term: 17 candidate_status { last_received { term: 16 index: 2220 } } ignore_live_leader: false dest_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e"
I20260812 06:36:48.978591  6046 raft_consensus.cc:3060] T 7065bb403e3047dd9a0853c49318ef1f P ec6c8d818e744cdbb6b0d15b321010e0 [term 16 FOLLOWER]: Advancing to term 17
I20260812 06:36:48.978622  8805 raft_consensus.cc:3060] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [term 16 FOLLOWER]: Advancing to term 17
I20260812 06:36:48.979439  7322 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8d90d61da62549208fe117b4384a469b in term 17.
I20260812 06:36:48.979667  8805 raft_consensus.cc:2410] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 8d90d61da62549208fe117b4384a469b for term 17 because replica has last-logged OpId of term: 16 index: 2228, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 2220.
I20260812 06:36:48.979717  6046 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P ec6c8d818e744cdbb6b0d15b321010e0 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8d90d61da62549208fe117b4384a469b in term 17.
I20260812 06:36:48.979960  6692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "8d90d61da62549208fe117b4384a469b" candidate_term: 17 candidate_status { last_received { term: 16 index: 2220 } } ignore_live_leader: false dest_uuid: "42895a9bcba64aa484f6509605035d02"
W20260812 06:36:48.980634  6150 leader_election.cc:336] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b [CANDIDATE]: Term 17 election: RPC error from VoteRequest() call to peer 2df4308c852240859ada53d216ab69ae (127.4.177.6:34269): Network error: Client connection negotiation failed: client connection to 127.4.177.6:34269: connect: Connection refused (error 111)
I20260812 06:36:48.980710  9423 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8d90d61da62549208fe117b4384a469b in term 17.
I20260812 06:36:48.980988  6150 leader_election.cc:304] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 7 responses out of 7 voters: 4 yes votes; 3 no votes. yes voters: 222b20de5a1145c6b8a11df8374e8790, 8d90d61da62549208fe117b4384a469b, b037d17b21f8422fa5e41b70eb4dcd08, ec6c8d818e744cdbb6b0d15b321010e0; no voters: 2df4308c852240859ada53d216ab69ae, 42895a9bcba64aa484f6509605035d02, 7962f587ccd842ad8a3f6ed9a5d7707e
I20260812 06:36:48.981144  9644 raft_consensus.cc:2804] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b [term 17 FOLLOWER]: Leader election won for term 17
I20260812 06:36:48.981359  9644 raft_consensus.cc:697] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b [term 17 LEADER]: Becoming Leader. State: Replica: 8d90d61da62549208fe117b4384a469b, State: Running, Role: LEADER
I20260812 06:36:48.981540  9644 consensus_queue.cc:237] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2219, Committed index: 2219, Last appended: 16.2220, Last appended by leader: 2220, Current term: 17, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df4308c852240859ada53d216ab69ae" member_type: VOTER last_known_addr { host: "127.4.177.6" port: 34269 } } peers { permanent_uuid: "42895a9bcba64aa484f6509605035d02" member_type: VOTER last_known_addr { host: "127.4.177.4" port: 38181 } } peers { permanent_uuid: "222b20de5a1145c6b8a11df8374e8790" member_type: VOTER last_known_addr { host: "127.4.177.5" port: 43521 } } peers { permanent_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 40897 } } peers { permanent_uuid: "8d90d61da62549208fe117b4384a469b" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 41539 } } peers { permanent_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" member_type: VOTER last_known_addr { host: "127.4.177.7" port: 46299 } } peers { permanent_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 41057 } }
I20260812 06:36:48.983508  9654 raft_consensus.cc:493] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 2df4308c852240859ada53d216ab69ae)
I20260812 06:36:48.983650  9654 raft_consensus.cc:515] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df4308c852240859ada53d216ab69ae" member_type: VOTER last_known_addr { host: "127.4.177.6" port: 34269 } } peers { permanent_uuid: "42895a9bcba64aa484f6509605035d02" member_type: VOTER last_known_addr { host: "127.4.177.4" port: 38181 } } peers { permanent_uuid: "222b20de5a1145c6b8a11df8374e8790" member_type: VOTER last_known_addr { host: "127.4.177.5" port: 43521 } } peers { permanent_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 40897 } } peers { permanent_uuid: "8d90d61da62549208fe117b4384a469b" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 41539 } } peers { permanent_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" member_type: VOTER last_known_addr { host: "127.4.177.7" port: 46299 } } peers { permanent_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 41057 } }
I20260812 06:36:48.984051  5810 catalog_manager.cc:5719] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b reported cstate change: term changed from 16 to 17, leader changed from 2df4308c852240859ada53d216ab69ae (127.4.177.6) to 8d90d61da62549208fe117b4384a469b (127.4.177.2). New cstate: current_term: 17 leader_uuid: "8d90d61da62549208fe117b4384a469b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df4308c852240859ada53d216ab69ae" member_type: VOTER last_known_addr { host: "127.4.177.6" port: 34269 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "42895a9bcba64aa484f6509605035d02" member_type: VOTER last_known_addr { host: "127.4.177.4" port: 38181 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "222b20de5a1145c6b8a11df8374e8790" member_type: VOTER last_known_addr { host: "127.4.177.5" port: 43521 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 40897 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8d90d61da62549208fe117b4384a469b" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 41539 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" member_type: VOTER last_known_addr { host: "127.4.177.7" port: 46299 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 41057 } health_report { overall_health: UNKNOWN } } }
I20260812 06:36:48.984499  9654 leader_election.cc:290] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 2df4308c852240859ada53d216ab69ae (127.4.177.6:34269), 222b20de5a1145c6b8a11df8374e8790 (127.4.177.5:43521), 7962f587ccd842ad8a3f6ed9a5d7707e (127.4.177.3:40897), 8d90d61da62549208fe117b4384a469b (127.4.177.2:41539), b037d17b21f8422fa5e41b70eb4dcd08 (127.4.177.7:46299), ec6c8d818e744cdbb6b0d15b321010e0 (127.4.177.1:41057)
I20260812 06:36:48.984871  7322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "42895a9bcba64aa484f6509605035d02" candidate_term: 17 candidate_status { last_received { term: 16 index: 2228 } } ignore_live_leader: false dest_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" is_pre_election: true
I20260812 06:36:48.985052  7322 raft_consensus.cc:2393] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 42895a9bcba64aa484f6509605035d02 in current term 17: Already voted for candidate 8d90d61da62549208fe117b4384a469b in this term.
I20260812 06:36:48.985534  8805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "42895a9bcba64aa484f6509605035d02" candidate_term: 17 candidate_status { last_received { term: 16 index: 2228 } } ignore_live_leader: false dest_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" is_pre_election: true
I20260812 06:36:48.985683  8805 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 42895a9bcba64aa484f6509605035d02 in term 17.
I20260812 06:36:48.985694  6046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "42895a9bcba64aa484f6509605035d02" candidate_term: 17 candidate_status { last_received { term: 16 index: 2228 } } ignore_live_leader: false dest_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" is_pre_election: true
I20260812 06:36:48.985818  6046 raft_consensus.cc:2393] T 7065bb403e3047dd9a0853c49318ef1f P ec6c8d818e744cdbb6b0d15b321010e0 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 42895a9bcba64aa484f6509605035d02 in current term 17: Already voted for candidate 8d90d61da62549208fe117b4384a469b in this term.
I20260812 06:36:48.985800  6245 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "42895a9bcba64aa484f6509605035d02" candidate_term: 17 candidate_status { last_received { term: 16 index: 2228 } } ignore_live_leader: false dest_uuid: "8d90d61da62549208fe117b4384a469b" is_pre_election: true
W20260812 06:36:48.986477  6597 leader_election.cc:336] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 2df4308c852240859ada53d216ab69ae (127.4.177.6:34269): Network error: Client connection negotiation failed: client connection to 127.4.177.6:34269: connect: Connection refused (error 111)
I20260812 06:36:48.986548  6597 leader_election.cc:304] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 6 responses out of 7 voters: 2 yes votes; 4 no votes. yes voters: 42895a9bcba64aa484f6509605035d02, 7962f587ccd842ad8a3f6ed9a5d7707e; no voters: 2df4308c852240859ada53d216ab69ae, 8d90d61da62549208fe117b4384a469b, b037d17b21f8422fa5e41b70eb4dcd08, ec6c8d818e744cdbb6b0d15b321010e0
I20260812 06:36:48.986692  9654 raft_consensus.cc:3060] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [term 16 FOLLOWER]: Advancing to term 17
I20260812 06:36:48.988598  9654 raft_consensus.cc:2749] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260812 06:36:48.990664  9423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "42895a9bcba64aa484f6509605035d02" candidate_term: 17 candidate_status { last_received { term: 16 index: 2228 } } ignore_live_leader: false dest_uuid: "222b20de5a1145c6b8a11df8374e8790" is_pre_election: true
I20260812 06:36:48.990856  9423 raft_consensus.cc:2393] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 42895a9bcba64aa484f6509605035d02 in current term 17: Already voted for candidate 8d90d61da62549208fe117b4384a469b in this term.
I20260812 06:36:49.038335  4804 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskCnZu03/build/debug/bin/kudu
/tmp/dist-test-taskCnZu03/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-5/wal
--fs_data_dirs=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-5/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-5/logs
--server_dump_info_path=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-5/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.177.6:34269
--local_ip_for_outbound_sockets=127.4.177.6
--tserver_master_addrs=127.4.177.62:42183
--webserver_port=42073
--webserver_interface=127.4.177.6
--builtin_ntp_servers=127.4.177.20:34335
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_cache_size_limit_mb=10
--consensus_rpc_timeout_ms=1000 with env {}
W20260812 06:36:49.146675  9679 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:36:49.146957  9679 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:36:49.147035  9679 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 06:36:49.150506  9679 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:36:49.150673  9679 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.177.6
I20260812 06:36:49.154675  9679 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-5/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.177.20:34335
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--consensus_rpc_timeout_ms=1000
--log_cache_size_limit_mb=10
--fs_data_dirs=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-5/data
--fs_wal_dir=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-5/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.177.6:34269
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-5/data/info.pb
--webserver_interface=127.4.177.6
--webserver_port=42073
--tserver_master_addrs=127.4.177.62:42183
--never_fsync=true
--heap_profile_path=/tmp/kudu.9679
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.177.6

Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 06:36:49.155855  9679 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:36:49.157034  9679 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 06:36:49.163921  9692 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:36:49.163937  9697 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:36:49.164047  9690 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:36:49.164258  9679 server_base.cc:1061] running on GCE node
I20260812 06:36:49.164762  9679 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:36:49.165349  9679 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:36:49.166543  9679 hybrid_clock.cc:648] HybridClock initialized: now 1786516609166522 us; error 36 us; skew 500 ppm
I20260812 06:36:49.168490  9679 webserver.cc:533] Webserver started at http://127.4.177.6:42073/ using document root <none> and password file <none>
I20260812 06:36:49.169145  9679 fs_manager.cc:362] Metadata directory not provided
I20260812 06:36:49.169265  9679 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:36:49.173019  9679 fs_manager.cc:714] Time spent opening directory manager: real 0.002s	user 0.002s	sys 0.000s
I20260812 06:36:49.175292  9706 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 06:36:49.176373  9679 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260812 06:36:49.176537  9679 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-5/data,/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-5/wal
uuid: "2df4308c852240859ada53d216ab69ae"
format_stamp: "Formatted at 2026-08-12 06:36:11 on dist-test-slave-jztv"
I20260812 06:36:49.176957  9679 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-5/wal
metadata directory: /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-5/wal
1 data directories: /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-5/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:36:49.192459  9679 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 06:36:49.193157  9679 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:36:49.193364  9679 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:36:49.193960  9679 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260812 06:36:49.195326  9719 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
I20260812 06:36:49.198710  9679 ts_tablet_manager.cc:585] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260812 06:36:49.198819  9679 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.004s	user 0.000s	sys 0.000s
I20260812 06:36:49.198915  9679 ts_tablet_manager.cc:600] Registering tablets (0/1 complete)
I20260812 06:36:49.200474  9679 ts_tablet_manager.cc:616] Registered 1 tablets
I20260812 06:36:49.200546  9679 ts_tablet_manager.cc:595] Time spent register tablets: real 0.002s	user 0.000s	sys 0.000s
I20260812 06:36:49.200891  9719 tablet_bootstrap.cc:492] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae: Bootstrap starting.
I20260812 06:36:49.212293  9679 rpc_server.cc:307] RPC server started. Bound to: 127.4.177.6:34269
I20260812 06:36:49.212327  9906 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.4.177.6:34269 every 8 connection(s)
I20260812 06:36:49.213303  9679 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-5/data/info.pb
I20260812 06:36:49.216073  4804 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskCnZu03/build/debug/bin/kudu as pid 9679
I20260812 06:36:49.228145  9907 heartbeater.cc:344] Connected to a master server at 127.4.177.62:42183
I20260812 06:36:49.228431  9907 heartbeater.cc:461] Registering TS with master...
I20260812 06:36:49.229038  9907 heartbeater.cc:507] Master 127.4.177.62:42183 requested a full tablet report, sending...
I20260812 06:36:49.231117  5810 ts_manager.cc:194] Re-registered known tserver with Master: 2df4308c852240859ada53d216ab69ae (127.4.177.6:34269)
I20260812 06:36:49.231980  5810 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.4.177.6:37505
I20260812 06:36:49.327478  9719 log.cc:826] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae: Log is configured to *not* fsync() on all Append() calls
I20260812 06:36:49.369836  6046 raft_consensus.cc:1275] T 7065bb403e3047dd9a0853c49318ef1f P ec6c8d818e744cdbb6b0d15b321010e0 [term 17 FOLLOWER]: Refusing update from remote peer 8d90d61da62549208fe117b4384a469b: Log matching property violated. Preceding OpId in replica: term: 16 index: 2161. Preceding OpId from leader: term: 17 index: 2221. (index mismatch)
I20260812 06:36:49.370368  9644 consensus_queue.cc:1048] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 41057 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2221, Last known committed idx: 2158, Time since last communication: 0.000s
W20260812 06:36:49.374429  6150 consensus_peers.cc:597] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b -> Peer 2df4308c852240859ada53d216ab69ae (127.4.177.6:34269): Couldn't send request to peer 2df4308c852240859ada53d216ab69ae. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260812 06:36:49.409391  6692 raft_consensus.cc:1275] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [term 17 FOLLOWER]: Refusing update from remote peer 8d90d61da62549208fe117b4384a469b: Log matching property violated. Preceding OpId in replica: term: 16 index: 2228. Preceding OpId from leader: term: 17 index: 2221. (term mismatch)
I20260812 06:36:49.409507  6692 pending_rounds.cc:85] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02: Aborting all ops after (but not including) 2220
I20260812 06:36:49.409540  6692 pending_rounds.cc:107] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02: Aborting uncommitted WRITE_OP operation due to leader change: 16.2221
I20260812 06:36:49.409827  6692 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:42408 (ReqId={client: 3067f3aef2a44cc4bb7f2ccd6acf83db, seq_no=2209, attempt_no=1}) took 1936 ms. Trace:
I20260812 06:36:49.409931  6692 rpcz_store.cc:276] 0812 06:36:47.473733 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 06:36:47.473870 (+   137us) service_pool.cc:224] Handling call
0812 06:36:49.409812 (+1935942us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260812 06:36:49.410202  6692 pending_rounds.cc:107] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02: Aborting uncommitted WRITE_OP operation due to leader change: 16.2222
I20260812 06:36:49.410401  6692 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:42408 (ReqId={client: 3067f3aef2a44cc4bb7f2ccd6acf83db, seq_no=2210, attempt_no=1}) took 1936 ms. Trace:
I20260812 06:36:49.410465  6692 rpcz_store.cc:276] 0812 06:36:47.473686 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 06:36:47.474060 (+   374us) service_pool.cc:224] Handling call
0812 06:36:49.410391 (+1936331us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260812 06:36:49.410612  6692 pending_rounds.cc:107] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02: Aborting uncommitted WRITE_OP operation due to leader change: 16.2223
I20260812 06:36:49.410759  6692 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:42408 (ReqId={client: 3067f3aef2a44cc4bb7f2ccd6acf83db, seq_no=2213, attempt_no=1}) took 1937 ms. Trace:
I20260812 06:36:49.410812  6692 rpcz_store.cc:276] 0812 06:36:47.473566 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 06:36:47.473933 (+   367us) service_pool.cc:224] Handling call
0812 06:36:49.410752 (+1936819us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260812 06:36:49.410975  6692 pending_rounds.cc:107] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02: Aborting uncommitted WRITE_OP operation due to leader change: 16.2224
I20260812 06:36:49.411123  6692 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:42408 (ReqId={client: 3067f3aef2a44cc4bb7f2ccd6acf83db, seq_no=2214, attempt_no=1}) took 1937 ms. Trace:
I20260812 06:36:49.411183  6692 rpcz_store.cc:276] 0812 06:36:47.473469 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 06:36:47.473954 (+   485us) service_pool.cc:224] Handling call
0812 06:36:49.411115 (+1937161us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260812 06:36:49.411336  6692 pending_rounds.cc:107] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02: Aborting uncommitted WRITE_OP operation due to leader change: 16.2225
I20260812 06:36:49.411487  6692 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:42408 (ReqId={client: 3067f3aef2a44cc4bb7f2ccd6acf83db, seq_no=2211, attempt_no=1}) took 1937 ms. Trace:
I20260812 06:36:49.411540  6692 rpcz_store.cc:276] 0812 06:36:47.473607 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 06:36:47.474019 (+   412us) service_pool.cc:224] Handling call
0812 06:36:49.411479 (+1937460us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260812 06:36:49.411674  6692 pending_rounds.cc:107] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02: Aborting uncommitted WRITE_OP operation due to leader change: 16.2226
I20260812 06:36:49.411823  6692 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:42408 (ReqId={client: 3067f3aef2a44cc4bb7f2ccd6acf83db, seq_no=2212, attempt_no=1}) took 1938 ms. Trace:
I20260812 06:36:49.411887  6692 rpcz_store.cc:276] 0812 06:36:47.473522 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 06:36:47.473735 (+   213us) service_pool.cc:224] Handling call
0812 06:36:49.411814 (+1938079us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260812 06:36:49.412034  6692 pending_rounds.cc:107] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02: Aborting uncommitted WRITE_OP operation due to leader change: 16.2227
I20260812 06:36:49.412180  6692 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:42408 (ReqId={client: 3067f3aef2a44cc4bb7f2ccd6acf83db, seq_no=2215, attempt_no=1}) took 1938 ms. Trace:
I20260812 06:36:49.412233  6692 rpcz_store.cc:276] 0812 06:36:47.473647 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 06:36:47.473978 (+   331us) service_pool.cc:224] Handling call
0812 06:36:49.412171 (+1938193us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260812 06:36:49.412380  6692 pending_rounds.cc:107] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02: Aborting uncommitted WRITE_OP operation due to leader change: 16.2228
I20260812 06:36:49.412532  6692 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:42408 (ReqId={client: 3067f3aef2a44cc4bb7f2ccd6acf83db, seq_no=2216, attempt_no=1}) took 1938 ms. Trace:
I20260812 06:36:49.412591  6692 rpcz_store.cc:276] 0812 06:36:47.473776 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 06:36:47.474379 (+   603us) service_pool.cc:224] Handling call
0812 06:36:49.412523 (+1938144us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260812 06:36:49.413283  9644 consensus_queue.cc:1048] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b [LEADER]: Connected to new peer: Peer: permanent_uuid: "42895a9bcba64aa484f6509605035d02" member_type: VOTER last_known_addr { host: "127.4.177.4" port: 38181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2221, Last known committed idx: 2220, Time since last communication: 0.000s
W20260812 06:36:49.415280  6631 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:42408: Illegal state: replica 42895a9bcba64aa484f6509605035d02 is not leader of this config: current role FOLLOWER
W20260812 06:36:49.415280  6620 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:42408: Illegal state: replica 42895a9bcba64aa484f6509605035d02 is not leader of this config: current role FOLLOWER
W20260812 06:36:49.416497  6620 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:42408: Illegal state: replica 42895a9bcba64aa484f6509605035d02 is not leader of this config: current role FOLLOWER
W20260812 06:36:49.417387  6620 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:42408: Illegal state: replica 42895a9bcba64aa484f6509605035d02 is not leader of this config: current role FOLLOWER
W20260812 06:36:49.418414  6620 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:42408: Illegal state: replica 42895a9bcba64aa484f6509605035d02 is not leader of this config: current role FOLLOWER
W20260812 06:36:49.418440  6631 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:42408: Illegal state: replica 42895a9bcba64aa484f6509605035d02 is not leader of this config: current role FOLLOWER
W20260812 06:36:49.418555  6629 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:42408: Illegal state: replica 42895a9bcba64aa484f6509605035d02 is not leader of this config: current role FOLLOWER
W20260812 06:36:49.420657  6629 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:42408: Illegal state: replica 42895a9bcba64aa484f6509605035d02 is not leader of this config: current role FOLLOWER
I20260812 06:36:49.510728  8805 raft_consensus.cc:1275] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [term 17 FOLLOWER]: Refusing update from remote peer 8d90d61da62549208fe117b4384a469b: Log matching property violated. Preceding OpId in replica: term: 16 index: 2228. Preceding OpId from leader: term: 17 index: 2221. (term mismatch)
I20260812 06:36:49.510847  8805 pending_rounds.cc:85] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e: Aborting all ops after (but not including) 2220
I20260812 06:36:49.510878  8805 pending_rounds.cc:107] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e: Aborting uncommitted WRITE_OP operation due to leader change: 16.2221
I20260812 06:36:49.511351  8805 pending_rounds.cc:107] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e: Aborting uncommitted WRITE_OP operation due to leader change: 16.2222
I20260812 06:36:49.511660  8805 pending_rounds.cc:107] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e: Aborting uncommitted WRITE_OP operation due to leader change: 16.2223
I20260812 06:36:49.512001  8805 pending_rounds.cc:107] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e: Aborting uncommitted WRITE_OP operation due to leader change: 16.2224
I20260812 06:36:49.512233  8805 pending_rounds.cc:107] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e: Aborting uncommitted WRITE_OP operation due to leader change: 16.2225
I20260812 06:36:49.512513  8805 pending_rounds.cc:107] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e: Aborting uncommitted WRITE_OP operation due to leader change: 16.2226
I20260812 06:36:49.512758  8805 pending_rounds.cc:107] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e: Aborting uncommitted WRITE_OP operation due to leader change: 16.2227
I20260812 06:36:49.513041  8805 pending_rounds.cc:107] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e: Aborting uncommitted WRITE_OP operation due to leader change: 16.2228
I20260812 06:36:49.513821  9644 consensus_queue.cc:1048] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 40897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2221, Last known committed idx: 2220, Time since last communication: 0.000s
I20260812 06:36:49.517994  7322 raft_consensus.cc:1275] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08 [term 17 FOLLOWER]: Refusing update from remote peer 8d90d61da62549208fe117b4384a469b: Log matching property violated. Preceding OpId in replica: term: 16 index: 2191. Preceding OpId from leader: term: 17 index: 2221. (index mismatch)
W20260812 06:36:49.519548  8742 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:48246: Illegal state: replica 7962f587ccd842ad8a3f6ed9a5d7707e is not leader of this config: current role FOLLOWER
W20260812 06:36:49.519996  8739 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:48246: Illegal state: replica 7962f587ccd842ad8a3f6ed9a5d7707e is not leader of this config: current role FOLLOWER
W20260812 06:36:49.520313  8741 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:48246: Illegal state: replica 7962f587ccd842ad8a3f6ed9a5d7707e is not leader of this config: current role FOLLOWER
W20260812 06:36:49.520581  8740 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:48246: Illegal state: replica 7962f587ccd842ad8a3f6ed9a5d7707e is not leader of this config: current role FOLLOWER
I20260812 06:36:49.522794  9423 raft_consensus.cc:1275] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [term 17 FOLLOWER]: Refusing update from remote peer 8d90d61da62549208fe117b4384a469b: Log matching property violated. Preceding OpId in replica: term: 14 index: 2073. Preceding OpId from leader: term: 17 index: 2221. (index mismatch)
W20260812 06:36:49.523783  8740 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:48246: Illegal state: replica 7962f587ccd842ad8a3f6ed9a5d7707e is not leader of this config: current role FOLLOWER
W20260812 06:36:49.524375  8739 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:48246: Illegal state: replica 7962f587ccd842ad8a3f6ed9a5d7707e is not leader of this config: current role FOLLOWER
W20260812 06:36:49.525059  8741 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:48246: Illegal state: replica 7962f587ccd842ad8a3f6ed9a5d7707e is not leader of this config: current role FOLLOWER
I20260812 06:36:49.521126  9952 consensus_queue.cc:1048] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" member_type: VOTER last_known_addr { host: "127.4.177.7" port: 46299 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2221, Last known committed idx: 2188, Time since last communication: 0.000s
W20260812 06:36:49.525890  8742 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:48246: Illegal state: replica 7962f587ccd842ad8a3f6ed9a5d7707e is not leader of this config: current role FOLLOWER
I20260812 06:36:49.526027  9947 consensus_queue.cc:1048] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b [LEADER]: Connected to new peer: Peer: permanent_uuid: "222b20de5a1145c6b8a11df8374e8790" member_type: VOTER last_known_addr { host: "127.4.177.5" port: 43521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2221, Last known committed idx: 2065, Time since last communication: 0.000s
I20260812 06:36:49.702412  9978 mvcc.cc:204] Tried to move back new op lower bound from 7317572021402189824 to 7317572016927584256. Current Snapshot: MvccSnapshot[applied={T|T < 7317572021313622016 or (T in {7317572021355044864,7317572021372137472,7317572021386510336,7317572021384122368})}]
I20260812 06:36:50.233947  9907 heartbeater.cc:499] Master 127.4.177.62:42183 was elected leader, sending a full tablet report...
I20260812 06:36:51.237859  9719 tablet_bootstrap.cc:492] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae: Bootstrap replayed 1/1 log segments. Stats: ops{read=2253 overwritten=25 applied=2220 ignored=0} inserts{seen=22090 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 8 replicates
I20260812 06:36:51.238489  9719 tablet_bootstrap.cc:492] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae: Bootstrap complete.
I20260812 06:36:51.239708  9719 ts_tablet_manager.cc:1403] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae: Time spent bootstrapping tablet: real 2.039s	user 1.668s	sys 0.091s
I20260812 06:36:51.241724  9719 raft_consensus.cc:359] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [term 16 FOLLOWER]: Replica starting. Triggering 8 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df4308c852240859ada53d216ab69ae" member_type: VOTER last_known_addr { host: "127.4.177.6" port: 34269 } } peers { permanent_uuid: "42895a9bcba64aa484f6509605035d02" member_type: VOTER last_known_addr { host: "127.4.177.4" port: 38181 } } peers { permanent_uuid: "222b20de5a1145c6b8a11df8374e8790" member_type: VOTER last_known_addr { host: "127.4.177.5" port: 43521 } } peers { permanent_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 40897 } } peers { permanent_uuid: "8d90d61da62549208fe117b4384a469b" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 41539 } } peers { permanent_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" member_type: VOTER last_known_addr { host: "127.4.177.7" port: 46299 } } peers { permanent_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 41057 } }
I20260812 06:36:51.243708  9719 raft_consensus.cc:740] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [term 16 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2df4308c852240859ada53d216ab69ae, State: Initialized, Role: FOLLOWER
I20260812 06:36:51.244160  9719 consensus_queue.cc:260] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2220, Last appended: 16.2228, Last appended by leader: 2228, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df4308c852240859ada53d216ab69ae" member_type: VOTER last_known_addr { host: "127.4.177.6" port: 34269 } } peers { permanent_uuid: "42895a9bcba64aa484f6509605035d02" member_type: VOTER last_known_addr { host: "127.4.177.4" port: 38181 } } peers { permanent_uuid: "222b20de5a1145c6b8a11df8374e8790" member_type: VOTER last_known_addr { host: "127.4.177.5" port: 43521 } } peers { permanent_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 40897 } } peers { permanent_uuid: "8d90d61da62549208fe117b4384a469b" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 41539 } } peers { permanent_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" member_type: VOTER last_known_addr { host: "127.4.177.7" port: 46299 } } peers { permanent_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 41057 } }
I20260812 06:36:51.244972  9719 ts_tablet_manager.cc:1434] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae: Time spent starting tablet: real 0.005s	user 0.006s	sys 0.000s
I20260812 06:36:51.605553 10014 raft_consensus.cc:493] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 8d90d61da62549208fe117b4384a469b)
I20260812 06:36:51.605719 10014 raft_consensus.cc:515] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df4308c852240859ada53d216ab69ae" member_type: VOTER last_known_addr { host: "127.4.177.6" port: 34269 } } peers { permanent_uuid: "42895a9bcba64aa484f6509605035d02" member_type: VOTER last_known_addr { host: "127.4.177.4" port: 38181 } } peers { permanent_uuid: "222b20de5a1145c6b8a11df8374e8790" member_type: VOTER last_known_addr { host: "127.4.177.5" port: 43521 } } peers { permanent_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 40897 } } peers { permanent_uuid: "8d90d61da62549208fe117b4384a469b" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 41539 } } peers { permanent_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" member_type: VOTER last_known_addr { host: "127.4.177.7" port: 46299 } } peers { permanent_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 41057 } }
I20260812 06:36:51.606644 10014 leader_election.cc:290] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 2df4308c852240859ada53d216ab69ae (127.4.177.6:34269), 42895a9bcba64aa484f6509605035d02 (127.4.177.4:38181), 222b20de5a1145c6b8a11df8374e8790 (127.4.177.5:43521), 7962f587ccd842ad8a3f6ed9a5d7707e (127.4.177.3:40897), 8d90d61da62549208fe117b4384a469b (127.4.177.2:41539), ec6c8d818e744cdbb6b0d15b321010e0 (127.4.177.1:41057)
I20260812 06:36:51.607528  6692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" candidate_term: 18 candidate_status { last_received { term: 17 index: 2353 } } ignore_live_leader: false dest_uuid: "42895a9bcba64aa484f6509605035d02" is_pre_election: true
I20260812 06:36:51.607641  9423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" candidate_term: 18 candidate_status { last_received { term: 17 index: 2353 } } ignore_live_leader: false dest_uuid: "222b20de5a1145c6b8a11df8374e8790" is_pre_election: true
I20260812 06:36:51.607729  6048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" candidate_term: 18 candidate_status { last_received { term: 17 index: 2353 } } ignore_live_leader: false dest_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" is_pre_election: true
I20260812 06:36:51.607710  8805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" candidate_term: 18 candidate_status { last_received { term: 17 index: 2353 } } ignore_live_leader: false dest_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" is_pre_election: true
I20260812 06:36:51.608295  7211 leader_election.cc:304] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 5 responses out of 7 voters: 1 yes votes; 4 no votes. yes voters: b037d17b21f8422fa5e41b70eb4dcd08; no voters: 222b20de5a1145c6b8a11df8374e8790, 42895a9bcba64aa484f6509605035d02, 7962f587ccd842ad8a3f6ed9a5d7707e, ec6c8d818e744cdbb6b0d15b321010e0
I20260812 06:36:51.608491 10014 raft_consensus.cc:2749] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260812 06:36:51.611734  9800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" candidate_term: 18 candidate_status { last_received { term: 17 index: 2353 } } ignore_live_leader: false dest_uuid: "2df4308c852240859ada53d216ab69ae" is_pre_election: true
I20260812 06:36:51.612069  9800 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b037d17b21f8422fa5e41b70eb4dcd08 in term 16.
I20260812 06:36:51.708197 10019 raft_consensus.cc:493] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 8d90d61da62549208fe117b4384a469b)
I20260812 06:36:51.708354 10019 raft_consensus.cc:515] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df4308c852240859ada53d216ab69ae" member_type: VOTER last_known_addr { host: "127.4.177.6" port: 34269 } } peers { permanent_uuid: "42895a9bcba64aa484f6509605035d02" member_type: VOTER last_known_addr { host: "127.4.177.4" port: 38181 } } peers { permanent_uuid: "222b20de5a1145c6b8a11df8374e8790" member_type: VOTER last_known_addr { host: "127.4.177.5" port: 43521 } } peers { permanent_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 40897 } } peers { permanent_uuid: "8d90d61da62549208fe117b4384a469b" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 41539 } } peers { permanent_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" member_type: VOTER last_known_addr { host: "127.4.177.7" port: 46299 } } peers { permanent_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 41057 } }
I20260812 06:36:51.709203 10019 leader_election.cc:290] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 2df4308c852240859ada53d216ab69ae (127.4.177.6:34269), 42895a9bcba64aa484f6509605035d02 (127.4.177.4:38181), 222b20de5a1145c6b8a11df8374e8790 (127.4.177.5:43521), 8d90d61da62549208fe117b4384a469b (127.4.177.2:41539), b037d17b21f8422fa5e41b70eb4dcd08 (127.4.177.7:46299), ec6c8d818e744cdbb6b0d15b321010e0 (127.4.177.1:41057)
I20260812 06:36:51.709803  7320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" candidate_term: 18 candidate_status { last_received { term: 17 index: 2385 } } ignore_live_leader: false dest_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" is_pre_election: true
I20260812 06:36:51.709991  7320 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7962f587ccd842ad8a3f6ed9a5d7707e in term 17.
I20260812 06:36:51.709992  9423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" candidate_term: 18 candidate_status { last_received { term: 17 index: 2385 } } ignore_live_leader: false dest_uuid: "222b20de5a1145c6b8a11df8374e8790" is_pre_election: true
I20260812 06:36:51.710007  6048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" candidate_term: 18 candidate_status { last_received { term: 17 index: 2385 } } ignore_live_leader: false dest_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" is_pre_election: true
I20260812 06:36:51.710186  6692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" candidate_term: 18 candidate_status { last_received { term: 17 index: 2385 } } ignore_live_leader: false dest_uuid: "42895a9bcba64aa484f6509605035d02" is_pre_election: true
I20260812 06:36:51.713307  9800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" candidate_term: 18 candidate_status { last_received { term: 17 index: 2385 } } ignore_live_leader: false dest_uuid: "2df4308c852240859ada53d216ab69ae" is_pre_election: true
I20260812 06:36:51.713490  9800 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7962f587ccd842ad8a3f6ed9a5d7707e in term 16.
I20260812 06:36:51.723569 10023 raft_consensus.cc:493] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 8d90d61da62549208fe117b4384a469b)
I20260812 06:36:51.723692 10023 raft_consensus.cc:515] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df4308c852240859ada53d216ab69ae" member_type: VOTER last_known_addr { host: "127.4.177.6" port: 34269 } } peers { permanent_uuid: "42895a9bcba64aa484f6509605035d02" member_type: VOTER last_known_addr { host: "127.4.177.4" port: 38181 } } peers { permanent_uuid: "222b20de5a1145c6b8a11df8374e8790" member_type: VOTER last_known_addr { host: "127.4.177.5" port: 43521 } } peers { permanent_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 40897 } } peers { permanent_uuid: "8d90d61da62549208fe117b4384a469b" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 41539 } } peers { permanent_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" member_type: VOTER last_known_addr { host: "127.4.177.7" port: 46299 } } peers { permanent_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 41057 } }
I20260812 06:36:51.724463 10023 leader_election.cc:290] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 2df4308c852240859ada53d216ab69ae (127.4.177.6:34269), 222b20de5a1145c6b8a11df8374e8790 (127.4.177.5:43521), 7962f587ccd842ad8a3f6ed9a5d7707e (127.4.177.3:40897), 8d90d61da62549208fe117b4384a469b (127.4.177.2:41539), b037d17b21f8422fa5e41b70eb4dcd08 (127.4.177.7:46299), ec6c8d818e744cdbb6b0d15b321010e0 (127.4.177.1:41057)
I20260812 06:36:51.725231  7320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "42895a9bcba64aa484f6509605035d02" candidate_term: 18 candidate_status { last_received { term: 17 index: 2393 } } ignore_live_leader: false dest_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" is_pre_election: true
I20260812 06:36:51.725419  7320 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 42895a9bcba64aa484f6509605035d02 in term 17.
I20260812 06:36:51.725411  8805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "42895a9bcba64aa484f6509605035d02" candidate_term: 18 candidate_status { last_received { term: 17 index: 2393 } } ignore_live_leader: false dest_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" is_pre_election: true
I20260812 06:36:51.725399  9423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "42895a9bcba64aa484f6509605035d02" candidate_term: 18 candidate_status { last_received { term: 17 index: 2393 } } ignore_live_leader: false dest_uuid: "222b20de5a1145c6b8a11df8374e8790" is_pre_election: true
I20260812 06:36:51.725548  8805 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 42895a9bcba64aa484f6509605035d02 in term 17.
I20260812 06:36:51.725819  6048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "42895a9bcba64aa484f6509605035d02" candidate_term: 18 candidate_status { last_received { term: 17 index: 2393 } } ignore_live_leader: false dest_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" is_pre_election: true
I20260812 06:36:51.728246  9800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "42895a9bcba64aa484f6509605035d02" candidate_term: 18 candidate_status { last_received { term: 17 index: 2393 } } ignore_live_leader: false dest_uuid: "2df4308c852240859ada53d216ab69ae" is_pre_election: true
I20260812 06:36:51.728370  9800 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 42895a9bcba64aa484f6509605035d02 in term 16.
I20260812 06:36:51.728647  6597 leader_election.cc:304] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 6 responses out of 7 voters: 4 yes votes; 2 no votes. yes voters: 2df4308c852240859ada53d216ab69ae, 42895a9bcba64aa484f6509605035d02, 7962f587ccd842ad8a3f6ed9a5d7707e, b037d17b21f8422fa5e41b70eb4dcd08; no voters: 222b20de5a1145c6b8a11df8374e8790, ec6c8d818e744cdbb6b0d15b321010e0
I20260812 06:36:51.728829 10023 raft_consensus.cc:2804] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260812 06:36:51.728893 10023 raft_consensus.cc:493] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 8d90d61da62549208fe117b4384a469b)
I20260812 06:36:51.728935 10023 raft_consensus.cc:3060] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [term 17 FOLLOWER]: Advancing to term 18
I20260812 06:36:51.729795 10023 raft_consensus.cc:515] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df4308c852240859ada53d216ab69ae" member_type: VOTER last_known_addr { host: "127.4.177.6" port: 34269 } } peers { permanent_uuid: "42895a9bcba64aa484f6509605035d02" member_type: VOTER last_known_addr { host: "127.4.177.4" port: 38181 } } peers { permanent_uuid: "222b20de5a1145c6b8a11df8374e8790" member_type: VOTER last_known_addr { host: "127.4.177.5" port: 43521 } } peers { permanent_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 40897 } } peers { permanent_uuid: "8d90d61da62549208fe117b4384a469b" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 41539 } } peers { permanent_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" member_type: VOTER last_known_addr { host: "127.4.177.7" port: 46299 } } peers { permanent_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 41057 } }
I20260812 06:36:51.730559 10023 leader_election.cc:290] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [CANDIDATE]: Term 18 election: Requested vote from peers 2df4308c852240859ada53d216ab69ae (127.4.177.6:34269), 222b20de5a1145c6b8a11df8374e8790 (127.4.177.5:43521), 7962f587ccd842ad8a3f6ed9a5d7707e (127.4.177.3:40897), 8d90d61da62549208fe117b4384a469b (127.4.177.2:41539), b037d17b21f8422fa5e41b70eb4dcd08 (127.4.177.7:46299), ec6c8d818e744cdbb6b0d15b321010e0 (127.4.177.1:41057)
I20260812 06:36:51.730536  9800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "42895a9bcba64aa484f6509605035d02" candidate_term: 18 candidate_status { last_received { term: 17 index: 2393 } } ignore_live_leader: false dest_uuid: "2df4308c852240859ada53d216ab69ae"
I20260812 06:36:51.730628  9423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "42895a9bcba64aa484f6509605035d02" candidate_term: 18 candidate_status { last_received { term: 17 index: 2393 } } ignore_live_leader: false dest_uuid: "222b20de5a1145c6b8a11df8374e8790"
I20260812 06:36:51.730664  9800 raft_consensus.cc:3060] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [term 16 FOLLOWER]: Advancing to term 18
I20260812 06:36:51.730883  8805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "42895a9bcba64aa484f6509605035d02" candidate_term: 18 candidate_status { last_received { term: 17 index: 2393 } } ignore_live_leader: false dest_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e"
I20260812 06:36:51.730903  7320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "42895a9bcba64aa484f6509605035d02" candidate_term: 18 candidate_status { last_received { term: 17 index: 2393 } } ignore_live_leader: false dest_uuid: "b037d17b21f8422fa5e41b70eb4dcd08"
I20260812 06:36:51.731016  7320 raft_consensus.cc:3060] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08 [term 17 FOLLOWER]: Advancing to term 18
I20260812 06:36:51.731016  8805 raft_consensus.cc:3060] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [term 17 FOLLOWER]: Advancing to term 18
I20260812 06:36:51.731985  8805 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 42895a9bcba64aa484f6509605035d02 in term 18.
I20260812 06:36:51.732087  7320 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 42895a9bcba64aa484f6509605035d02 in term 18.
I20260812 06:36:51.732543  9800 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 42895a9bcba64aa484f6509605035d02 in term 18.
I20260812 06:36:51.732633  6048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "42895a9bcba64aa484f6509605035d02" candidate_term: 18 candidate_status { last_received { term: 17 index: 2393 } } ignore_live_leader: false dest_uuid: "ec6c8d818e744cdbb6b0d15b321010e0"
I20260812 06:36:51.732869  6597 leader_election.cc:304] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 5 responses out of 7 voters: 4 yes votes; 1 no votes. yes voters: 2df4308c852240859ada53d216ab69ae, 42895a9bcba64aa484f6509605035d02, 7962f587ccd842ad8a3f6ed9a5d7707e, b037d17b21f8422fa5e41b70eb4dcd08; no voters: 222b20de5a1145c6b8a11df8374e8790
I20260812 06:36:51.733068 10023 raft_consensus.cc:2804] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [term 18 FOLLOWER]: Leader election won for term 18
I20260812 06:36:51.733251 10023 raft_consensus.cc:697] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [term 18 LEADER]: Becoming Leader. State: Replica: 42895a9bcba64aa484f6509605035d02, State: Running, Role: LEADER
I20260812 06:36:51.733435 10023 consensus_queue.cc:237] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2385, Committed index: 2385, Last appended: 17.2393, Last appended by leader: 2393, Current term: 18, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df4308c852240859ada53d216ab69ae" member_type: VOTER last_known_addr { host: "127.4.177.6" port: 34269 } } peers { permanent_uuid: "42895a9bcba64aa484f6509605035d02" member_type: VOTER last_known_addr { host: "127.4.177.4" port: 38181 } } peers { permanent_uuid: "222b20de5a1145c6b8a11df8374e8790" member_type: VOTER last_known_addr { host: "127.4.177.5" port: 43521 } } peers { permanent_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 40897 } } peers { permanent_uuid: "8d90d61da62549208fe117b4384a469b" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 41539 } } peers { permanent_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" member_type: VOTER last_known_addr { host: "127.4.177.7" port: 46299 } } peers { permanent_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 41057 } }
I20260812 06:36:51.735657  5808 catalog_manager.cc:5719] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 reported cstate change: term changed from 17 to 18, leader changed from 8d90d61da62549208fe117b4384a469b (127.4.177.2) to 42895a9bcba64aa484f6509605035d02 (127.4.177.4). New cstate: current_term: 18 leader_uuid: "42895a9bcba64aa484f6509605035d02" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df4308c852240859ada53d216ab69ae" member_type: VOTER last_known_addr { host: "127.4.177.6" port: 34269 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "42895a9bcba64aa484f6509605035d02" member_type: VOTER last_known_addr { host: "127.4.177.4" port: 38181 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "222b20de5a1145c6b8a11df8374e8790" member_type: VOTER last_known_addr { host: "127.4.177.5" port: 43521 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 40897 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8d90d61da62549208fe117b4384a469b" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 41539 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" member_type: VOTER last_known_addr { host: "127.4.177.7" port: 46299 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 41057 } health_report { overall_health: UNKNOWN } } }
W20260812 06:36:51.914753  6150 connection.cc:256] RPC call timeout handler was delayed by -1.0222184450831264s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1s for a response
I20260812 06:36:51.916203  6247 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" candidate_term: 18 candidate_status { last_received { term: 17 index: 2385 } } ignore_live_leader: false dest_uuid: "8d90d61da62549208fe117b4384a469b" is_pre_election: true
I20260812 06:36:51.916671  8698 leader_election.cc:304] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 7 responses out of 7 voters: 3 yes votes; 4 no votes. yes voters: 2df4308c852240859ada53d216ab69ae, 7962f587ccd842ad8a3f6ed9a5d7707e, b037d17b21f8422fa5e41b70eb4dcd08; no voters: 222b20de5a1145c6b8a11df8374e8790, 42895a9bcba64aa484f6509605035d02, 8d90d61da62549208fe117b4384a469b, ec6c8d818e744cdbb6b0d15b321010e0
I20260812 06:36:51.917006 10019 raft_consensus.cc:2749] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260812 06:36:51.917271  6245 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "42895a9bcba64aa484f6509605035d02" candidate_term: 18 candidate_status { last_received { term: 17 index: 2393 } } ignore_live_leader: false dest_uuid: "8d90d61da62549208fe117b4384a469b" is_pre_election: true
I20260812 06:36:51.917481  6692 raft_consensus.cc:1240] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [term 18 LEADER]: Rejecting Update request from peer 8d90d61da62549208fe117b4384a469b for earlier term 17. Current term is 18. Ops: []
I20260812 06:36:51.919095  9983 consensus_queue.cc:1059] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "42895a9bcba64aa484f6509605035d02" member_type: VOTER last_known_addr { host: "127.4.177.4" port: 38181 }, Status: INVALID_TERM, Last received: 17.2393, Next index: 2394, Last known committed idx: 2385, Time since last communication: 0.000s
I20260812 06:36:51.919225  9983 consensus_queue.cc:1243] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b [LEADER]: Peer 42895a9bcba64aa484f6509605035d02 log is divergent from this leader: its last log entry 18.2394 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2385
I20260812 06:36:51.919284  9983 raft_consensus.cc:3055] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b [term 17 LEADER]: Stepping down as leader of term 17
I20260812 06:36:51.919317  9983 raft_consensus.cc:740] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 8d90d61da62549208fe117b4384a469b, State: Running, Role: LEADER
I20260812 06:36:51.919417  8805 raft_consensus.cc:1240] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [term 18 FOLLOWER]: Rejecting Update request from peer 8d90d61da62549208fe117b4384a469b for earlier term 17. Current term is 18. Ops: [17.2386-17.2393]
I20260812 06:36:51.919479  9983 consensus_queue.cc:260] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2385, Committed index: 2385, Last appended: 17.2393, Last appended by leader: 2393, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df4308c852240859ada53d216ab69ae" member_type: VOTER last_known_addr { host: "127.4.177.6" port: 34269 } } peers { permanent_uuid: "42895a9bcba64aa484f6509605035d02" member_type: VOTER last_known_addr { host: "127.4.177.4" port: 38181 } } peers { permanent_uuid: "222b20de5a1145c6b8a11df8374e8790" member_type: VOTER last_known_addr { host: "127.4.177.5" port: 43521 } } peers { permanent_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 40897 } } peers { permanent_uuid: "8d90d61da62549208fe117b4384a469b" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 41539 } } peers { permanent_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" member_type: VOTER last_known_addr { host: "127.4.177.7" port: 46299 } } peers { permanent_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 41057 } }
I20260812 06:36:51.919656  9983 raft_consensus.cc:3060] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b [term 17 FOLLOWER]: Advancing to term 18
I20260812 06:36:51.919911  7320 raft_consensus.cc:1240] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08 [term 18 FOLLOWER]: Rejecting Update request from peer 8d90d61da62549208fe117b4384a469b for earlier term 17. Current term is 18. Ops: [17.2354-17.2393]
I20260812 06:36:51.920807  6242 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" candidate_term: 18 candidate_status { last_received { term: 17 index: 2353 } } ignore_live_leader: false dest_uuid: "8d90d61da62549208fe117b4384a469b" is_pre_election: true
I20260812 06:36:51.920935  6242 raft_consensus.cc:2410] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b037d17b21f8422fa5e41b70eb4dcd08 for term 18 because replica has last-logged OpId of term: 17 index: 2393, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 2353.
I20260812 06:36:51.921440  6238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "42895a9bcba64aa484f6509605035d02" candidate_term: 18 candidate_status { last_received { term: 17 index: 2393 } } ignore_live_leader: false dest_uuid: "8d90d61da62549208fe117b4384a469b"
I20260812 06:36:51.922447  6238 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 42895a9bcba64aa484f6509605035d02 in term 18.
I20260812 06:36:51.922884  9800 raft_consensus.cc:1240] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [term 18 FOLLOWER]: Rejecting Update request from peer 8d90d61da62549208fe117b4384a469b for earlier term 17. Current term is 18. Ops: [17.2221-17.2393]
I20260812 06:36:52.112495  9423 raft_consensus.cc:3060] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [term 17 FOLLOWER]: Advancing to term 18
I20260812 06:36:52.113586  9423 raft_consensus.cc:1275] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [term 18 FOLLOWER]: Refusing update from remote peer 42895a9bcba64aa484f6509605035d02: Log matching property violated. Preceding OpId in replica: term: 17 index: 2393. Preceding OpId from leader: term: 18 index: 2394. (index mismatch)
I20260812 06:36:52.114131 10023 consensus_queue.cc:1048] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [LEADER]: Connected to new peer: Peer: permanent_uuid: "222b20de5a1145c6b8a11df8374e8790" member_type: VOTER last_known_addr { host: "127.4.177.5" port: 43521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2394, Last known committed idx: 2385, Time since last communication: 0.000s
I20260812 06:36:52.297571  6048 raft_consensus.cc:3060] T 7065bb403e3047dd9a0853c49318ef1f P ec6c8d818e744cdbb6b0d15b321010e0 [term 17 FOLLOWER]: Advancing to term 18
I20260812 06:36:52.298699  6048 raft_consensus.cc:1275] T 7065bb403e3047dd9a0853c49318ef1f P ec6c8d818e744cdbb6b0d15b321010e0 [term 18 FOLLOWER]: Refusing update from remote peer 42895a9bcba64aa484f6509605035d02: Log matching property violated. Preceding OpId in replica: term: 17 index: 2393. Preceding OpId from leader: term: 18 index: 2394. (index mismatch)
I20260812 06:36:52.299218 10023 consensus_queue.cc:1048] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 41057 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2394, Last known committed idx: 2385, Time since last communication: 0.000s
I20260812 06:36:52.315902  9800 raft_consensus.cc:1275] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [term 18 FOLLOWER]: Refusing update from remote peer 42895a9bcba64aa484f6509605035d02: Log matching property violated. Preceding OpId in replica: term: 16 index: 2228. Preceding OpId from leader: term: 18 index: 2394. (index mismatch)
I20260812 06:36:52.316439 10023 consensus_queue.cc:1048] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2df4308c852240859ada53d216ab69ae" member_type: VOTER last_known_addr { host: "127.4.177.6" port: 34269 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2394, Last known committed idx: 2220, Time since last communication: 0.000s
I20260812 06:36:52.316619 10023 consensus_queue.cc:1243] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [LEADER]: Peer 2df4308c852240859ada53d216ab69ae log is divergent from this leader: its last log entry 16.2228 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2220
I20260812 06:36:52.320700  9800 pending_rounds.cc:85] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae: Aborting all ops after (but not including) 2220
I20260812 06:36:52.320827  9800 pending_rounds.cc:107] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae: Aborting uncommitted WRITE_OP operation due to leader change: 16.2221
I20260812 06:36:52.321333  9800 pending_rounds.cc:107] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae: Aborting uncommitted WRITE_OP operation due to leader change: 16.2222
I20260812 06:36:52.321547  9800 pending_rounds.cc:107] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae: Aborting uncommitted WRITE_OP operation due to leader change: 16.2223
I20260812 06:36:52.321719  9800 pending_rounds.cc:107] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae: Aborting uncommitted WRITE_OP operation due to leader change: 16.2224
I20260812 06:36:52.321885  9800 pending_rounds.cc:107] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae: Aborting uncommitted WRITE_OP operation due to leader change: 16.2225
I20260812 06:36:52.322053  9800 pending_rounds.cc:107] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae: Aborting uncommitted WRITE_OP operation due to leader change: 16.2226
I20260812 06:36:52.322266  9800 pending_rounds.cc:107] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae: Aborting uncommitted WRITE_OP operation due to leader change: 16.2227
I20260812 06:36:52.322445  9800 pending_rounds.cc:107] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae: Aborting uncommitted WRITE_OP operation due to leader change: 16.2228
I20260812 06:36:52.337447  6236 raft_consensus.cc:1275] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b [term 18 FOLLOWER]: Refusing update from remote peer 42895a9bcba64aa484f6509605035d02: Log matching property violated. Preceding OpId in replica: term: 17 index: 2393. Preceding OpId from leader: term: 18 index: 2394. (index mismatch)
I20260812 06:36:52.338016 10023 consensus_queue.cc:1048] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8d90d61da62549208fe117b4384a469b" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 41539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2394, Last known committed idx: 2385, Time since last communication: 0.000s
I20260812 06:36:52.339262  7320 raft_consensus.cc:1275] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08 [term 18 FOLLOWER]: Refusing update from remote peer 42895a9bcba64aa484f6509605035d02: Log matching property violated. Preceding OpId in replica: term: 17 index: 2353. Preceding OpId from leader: term: 18 index: 2394. (index mismatch)
I20260812 06:36:52.339747 10023 consensus_queue.cc:1048] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" member_type: VOTER last_known_addr { host: "127.4.177.7" port: 46299 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2394, Last known committed idx: 2352, Time since last communication: 0.000s
I20260812 06:36:52.346890  8805 raft_consensus.cc:1275] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [term 18 FOLLOWER]: Refusing update from remote peer 42895a9bcba64aa484f6509605035d02: Log matching property violated. Preceding OpId in replica: term: 17 index: 2385. Preceding OpId from leader: term: 18 index: 2394. (index mismatch)
I20260812 06:36:52.356921 10023 consensus_queue.cc:1048] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 40897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2394, Last known committed idx: 2383, Time since last communication: 0.000s
I20260812 06:36:52.372131 10118 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:42054 (ReqId={client: 3067f3aef2a44cc4bb7f2ccd6acf83db, seq_no=2373, attempt_no=0}) took 2167 ms. Trace:
I20260812 06:36:52.372243 10118 rpcz_store.cc:276] 0812 06:36:50.204901 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 06:36:50.204948 (+    47us) service_pool.cc:224] Handling call
0812 06:36:52.372089 (+2167141us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0812 06:36:50.208227 (+     0us) write_op.cc:183] PREPARE: starting on tablet 7065bb403e3047dd9a0853c49318ef1f
0812 06:36:50.208307 (+    80us) write_op.cc:432] Acquiring schema lock in shared mode
0812 06:36:50.208312 (+     5us) write_op.cc:435] Acquired schema lock
0812 06:36:50.208313 (+     1us) tablet.cc:662] Decoding operations
0812 06:36:50.208560 (+   247us) write_op.cc:620] Acquiring the partition lock for write op
0812 06:36:50.208566 (+     6us) write_op.cc:641] Partition lock acquired for write op
0812 06:36:50.208570 (+     4us) tablet.cc:685] Acquiring locks for 10 operations
0812 06:36:50.208635 (+    65us) tablet.cc:701] Row locks acquired
0812 06:36:50.208637 (+     2us) write_op.cc:260] PREPARE: finished
0812 06:36:50.208664 (+    27us) write_op.cc:270] Start()
0812 06:36:50.208682 (+    18us) write_op.cc:276] Timestamp: P: 1786516610208631 usec, L: 0
0812 06:36:50.208685 (+     3us) op_driver.cc:348] REPLICATION: starting
0812 06:36:50.209158 (+   473us) log.cc:844] Serialized 696 byte log entry
0812 06:36:52.358780 (+2149622us) op_driver.cc:464] REPLICATION: finished
0812 06:36:52.371490 (+ 12710us) write_op.cc:301] APPLY: starting
0812 06:36:52.371507 (+    17us) tablet.cc:1367] starting BulkCheckPresence
0812 06:36:52.371542 (+    35us) tablet.cc:1370] finished BulkCheckPresence
0812 06:36:52.371560 (+    18us) tablet.cc:1372] starting ApplyRowOperation cycle
0812 06:36:52.371759 (+   199us) tablet.cc:1383] finished ApplyRowOperation cycle
0812 06:36:52.371785 (+    26us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0812 06:36:52.371789 (+     4us) write_op.cc:312] APPLY: finished
0812 06:36:52.371845 (+    56us) log.cc:844] Serialized 77 byte log entry
0812 06:36:52.371985 (+   140us) write_op.cc:489] Releasing partition, row and schema locks
0812 06:36:52.372005 (+    20us) write_op.cc:454] Released schema lock
0812 06:36:52.372012 (+     7us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":12645,"num_ops":10,"prepare.queue_time_us":2976,"prepare.run_cpu_time_us":597,"prepare.run_wall_time_us":1000,"raft.queue_time_us":902,"raft.run_cpu_time_us":6,"raft.run_wall_time_us":7,"replication_time_us":2150072,"spinlock_wait_cycles":816896,"thread_start_us":95,"threads_started":1}]]}
I20260812 06:36:52.374082 10118 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:42054 (ReqId={client: 3067f3aef2a44cc4bb7f2ccd6acf83db, seq_no=2375, attempt_no=0}) took 2168 ms. Trace:
W20260812 06:36:52.374264  6179 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:42054: Illegal state: replica 8d90d61da62549208fe117b4384a469b is not leader of this config: current role FOLLOWER
I20260812 06:36:52.374986 10123 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:42054 (ReqId={client: 3067f3aef2a44cc4bb7f2ccd6acf83db, seq_no=2374, attempt_no=0}) took 2169 ms. Trace:
I20260812 06:36:52.375103 10123 rpcz_store.cc:276] 0812 06:36:50.205551 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 06:36:50.205925 (+   374us) service_pool.cc:224] Handling call
0812 06:36:52.374958 (+2169033us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0812 06:36:50.211570 (+     0us) write_op.cc:183] PREPARE: starting on tablet 7065bb403e3047dd9a0853c49318ef1f
0812 06:36:50.211641 (+    71us) write_op.cc:432] Acquiring schema lock in shared mode
0812 06:36:50.211645 (+     4us) write_op.cc:435] Acquired schema lock
0812 06:36:50.211647 (+     2us) tablet.cc:662] Decoding operations
0812 06:36:50.211730 (+    83us) write_op.cc:620] Acquiring the partition lock for write op
0812 06:36:50.211735 (+     5us) write_op.cc:641] Partition lock acquired for write op
0812 06:36:50.211738 (+     3us) tablet.cc:685] Acquiring locks for 10 operations
0812 06:36:50.211797 (+    59us) tablet.cc:701] Row locks acquired
0812 06:36:50.211798 (+     1us) write_op.cc:260] PREPARE: finished
0812 06:36:50.211825 (+    27us) write_op.cc:270] Start()
0812 06:36:50.211845 (+    20us) write_op.cc:276] Timestamp: P: 1786516610211790 usec, L: 0
0812 06:36:50.211848 (+     3us) op_driver.cc:348] REPLICATION: starting
0812 06:36:50.211964 (+   116us) log.cc:844] Serialized 686 byte log entry
0812 06:36:52.359037 (+2147073us) op_driver.cc:464] REPLICATION: finished
0812 06:36:52.374349 (+ 15312us) write_op.cc:301] APPLY: starting
0812 06:36:52.374360 (+    11us) tablet.cc:1367] starting BulkCheckPresence
0812 06:36:52.374398 (+    38us) tablet.cc:1370] finished BulkCheckPresence
0812 06:36:52.374410 (+    12us) tablet.cc:1372] starting ApplyRowOperation cycle
0812 06:36:52.374573 (+   163us) tablet.cc:1383] finished ApplyRowOperation cycle
0812 06:36:52.374597 (+    24us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0812 06:36:52.374602 (+     5us) write_op.cc:312] APPLY: finished
0812 06:36:52.374714 (+   112us) log.cc:844] Serialized 77 byte log entry
0812 06:36:52.374866 (+   152us) write_op.cc:489] Releasing partition, row and schema locks
0812 06:36:52.374879 (+    13us) write_op.cc:454] Released schema lock
0812 06:36:52.374884 (+     5us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":15272,"num_ops":10,"prepare.queue_time_us":5443,"prepare.run_cpu_time_us":460,"prepare.run_wall_time_us":1059,"raft.queue_time_us":93,"raft.run_cpu_time_us":3,"raft.run_wall_time_us":4,"replication_time_us":2147176,"thread_start_us":63,"threads_started":1}]]}
W20260812 06:36:52.377125  6179 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:42054: Illegal state: replica 8d90d61da62549208fe117b4384a469b is not leader of this config: current role FOLLOWER
I20260812 06:36:52.377782 10123 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:42054 (ReqId={client: 3067f3aef2a44cc4bb7f2ccd6acf83db, seq_no=2378, attempt_no=0}) took 2171 ms. Trace:
I20260812 06:36:52.377895 10123 rpcz_store.cc:276] 0812 06:36:50.206601 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 06:36:50.206638 (+    37us) service_pool.cc:224] Handling call
0812 06:36:52.377772 (+2171134us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0812 06:36:50.212657 (+     0us) write_op.cc:183] PREPARE: starting on tablet 7065bb403e3047dd9a0853c49318ef1f
0812 06:36:50.212720 (+    63us) write_op.cc:432] Acquiring schema lock in shared mode
0812 06:36:50.212723 (+     3us) write_op.cc:435] Acquired schema lock
0812 06:36:50.212725 (+     2us) tablet.cc:662] Decoding operations
0812 06:36:50.212799 (+    74us) write_op.cc:620] Acquiring the partition lock for write op
0812 06:36:50.212804 (+     5us) write_op.cc:641] Partition lock acquired for write op
0812 06:36:50.212807 (+     3us) tablet.cc:685] Acquiring locks for 10 operations
0812 06:36:50.212864 (+    57us) tablet.cc:701] Row locks acquired
0812 06:36:50.212866 (+     2us) write_op.cc:260] PREPARE: finished
0812 06:36:50.212891 (+    25us) write_op.cc:270] Start()
0812 06:36:50.212908 (+    17us) write_op.cc:276] Timestamp: P: 1786516610212859 usec, L: 0
0812 06:36:50.212909 (+     1us) op_driver.cc:348] REPLICATION: starting
0812 06:36:50.213005 (+    96us) log.cc:844] Serialized 696 byte log entry
0812 06:36:52.359147 (+2146142us) op_driver.cc:464] REPLICATION: finished
0812 06:36:52.375950 (+ 16803us) write_op.cc:301] APPLY: starting
0812 06:36:52.375961 (+    11us) tablet.cc:1367] starting BulkCheckPresence
0812 06:36:52.375989 (+    28us) tablet.cc:1370] finished BulkCheckPresence
0812 06:36:52.376000 (+    11us) tablet.cc:1372] starting ApplyRowOperation cycle
0812 06:36:52.376244 (+   244us) tablet.cc:1383] finished ApplyRowOperation cycle
0812 06:36:52.376271 (+    27us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0812 06:36:52.376276 (+     5us) write_op.cc:312] APPLY: finished
0812 06:36:52.376327 (+    51us) log.cc:844] Serialized 77 byte log entry
0812 06:36:52.377704 (+  1377us) write_op.cc:489] Releasing partition, row and schema locks
0812 06:36:52.377716 (+    12us) write_op.cc:454] Released schema lock
0812 06:36:52.377722 (+     6us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":16772,"num_ops":10,"prepare.queue_time_us":5789,"prepare.run_cpu_time_us":417,"prepare.run_wall_time_us":458,"raft.queue_time_us":30,"raft.run_cpu_time_us":4,"raft.run_wall_time_us":4,"replication_time_us":2146223,"thread_start_us":81,"threads_started":1}]]}
I20260812 06:36:52.379702 10123 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:42054 (ReqId={client: 3067f3aef2a44cc4bb7f2ccd6acf83db, seq_no=2377, attempt_no=0}) took 2173 ms. Trace:
I20260812 06:36:52.379812 10123 rpcz_store.cc:276] 0812 06:36:50.206545 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 06:36:50.206886 (+   341us) service_pool.cc:224] Handling call
0812 06:36:52.379692 (+2172806us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0812 06:36:50.213129 (+     0us) write_op.cc:183] PREPARE: starting on tablet 7065bb403e3047dd9a0853c49318ef1f
0812 06:36:50.213195 (+    66us) write_op.cc:432] Acquiring schema lock in shared mode
0812 06:36:50.213198 (+     3us) write_op.cc:435] Acquired schema lock
0812 06:36:50.213199 (+     1us) tablet.cc:662] Decoding operations
0812 06:36:50.213277 (+    78us) write_op.cc:620] Acquiring the partition lock for write op
0812 06:36:50.213282 (+     5us) write_op.cc:641] Partition lock acquired for write op
0812 06:36:50.213285 (+     3us) tablet.cc:685] Acquiring locks for 10 operations
0812 06:36:50.213339 (+    54us) tablet.cc:701] Row locks acquired
0812 06:36:50.213341 (+     2us) write_op.cc:260] PREPARE: finished
0812 06:36:50.213364 (+    23us) write_op.cc:270] Start()
0812 06:36:50.213379 (+    15us) write_op.cc:276] Timestamp: P: 1786516610213332 usec, L: 0
0812 06:36:50.213381 (+     2us) op_driver.cc:348] REPLICATION: starting
0812 06:36:50.213481 (+   100us) log.cc:844] Serialized 696 byte log entry
0812 06:36:52.359280 (+2145799us) op_driver.cc:464] REPLICATION: finished
0812 06:36:52.378995 (+ 19715us) write_op.cc:301] APPLY: starting
0812 06:36:52.379007 (+    12us) tablet.cc:1367] starting BulkCheckPresence
0812 06:36:52.379035 (+    28us) tablet.cc:1370] finished BulkCheckPresence
0812 06:36:52.379045 (+    10us) tablet.cc:1372] starting ApplyRowOperation cycle
0812 06:36:52.379213 (+   168us) tablet.cc:1383] finished ApplyRowOperation cycle
0812 06:36:52.379240 (+    27us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0812 06:36:52.379244 (+     4us) write_op.cc:312] APPLY: finished
0812 06:36:52.379300 (+    56us) log.cc:844] Serialized 77 byte log entry
0812 06:36:52.379623 (+   323us) write_op.cc:489] Releasing partition, row and schema locks
0812 06:36:52.379634 (+    11us) write_op.cc:454] Released schema lock
0812 06:36:52.379639 (+     5us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":19685,"num_ops":10,"prepare.queue_time_us":6031,"prepare.run_cpu_time_us":416,"prepare.run_wall_time_us":457,"raft.queue_time_us":28,"raft.run_cpu_time_us":4,"raft.run_wall_time_us":5,"replication_time_us":2145884}]]}
I20260812 06:36:52.384483 10125 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:42054 (ReqId={client: 3067f3aef2a44cc4bb7f2ccd6acf83db, seq_no=2379, attempt_no=0}) took 2174 ms. Trace:
I20260812 06:36:52.387426 10124 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:42054 (ReqId={client: 3067f3aef2a44cc4bb7f2ccd6acf83db, seq_no=2380, attempt_no=0}) took 2176 ms. Trace:
I20260812 06:36:52.406423 10125 rpcz_store.cc:276] 0812 06:36:50.210502 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 06:36:50.210552 (+    50us) service_pool.cc:224] Handling call
0812 06:36:52.384467 (+2173915us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0812 06:36:50.214062 (+     0us) write_op.cc:183] PREPARE: starting on tablet 7065bb403e3047dd9a0853c49318ef1f
0812 06:36:50.214126 (+    64us) write_op.cc:432] Acquiring schema lock in shared mode
0812 06:36:50.214129 (+     3us) write_op.cc:435] Acquired schema lock
0812 06:36:50.214130 (+     1us) tablet.cc:662] Decoding operations
0812 06:36:50.214207 (+    77us) write_op.cc:620] Acquiring the partition lock for write op
0812 06:36:50.214245 (+    38us) write_op.cc:641] Partition lock acquired for write op
0812 06:36:50.214248 (+     3us) tablet.cc:685] Acquiring locks for 10 operations
0812 06:36:50.214307 (+    59us) tablet.cc:701] Row locks acquired
0812 06:36:50.214309 (+     2us) write_op.cc:260] PREPARE: finished
0812 06:36:50.214333 (+    24us) write_op.cc:270] Start()
0812 06:36:50.214349 (+    16us) write_op.cc:276] Timestamp: P: 1786516610214301 usec, L: 0
0812 06:36:50.214350 (+     1us) op_driver.cc:348] REPLICATION: starting
0812 06:36:50.214447 (+    97us) log.cc:844] Serialized 696 byte log entry
0812 06:36:52.359361 (+2144914us) op_driver.cc:464] REPLICATION: finished
0812 06:36:52.383982 (+ 24621us) write_op.cc:301] APPLY: starting
0812 06:36:52.383994 (+    12us) tablet.cc:1367] starting BulkCheckPresence
0812 06:36:52.384023 (+    29us) tablet.cc:1370] finished BulkCheckPresence
0812 06:36:52.384036 (+    13us) tablet.cc:1372] starting ApplyRowOperation cycle
0812 06:36:52.384233 (+   197us) tablet.cc:1383] finished ApplyRowOperation cycle
0812 06:36:52.384260 (+    27us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0812 06:36:52.384264 (+     4us) write_op.cc:312] APPLY: finished
0812 06:36:52.384325 (+    61us) log.cc:844] Serialized 77 byte log entry
0812 06:36:52.384377 (+    52us) write_op.cc:489] Releasing partition, row and schema locks
0812 06:36:52.384390 (+    13us) write_op.cc:454] Released schema lock
0812 06:36:52.384395 (+     5us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":24569,"num_ops":10,"prepare.queue_time_us":3288,"prepare.run_cpu_time_us":445,"prepare.run_wall_time_us":488,"raft.queue_time_us":26,"raft.run_cpu_time_us":4,"raft.run_wall_time_us":5,"replication_time_us":2144978}]]}
I20260812 06:36:52.406442 10124 rpcz_store.cc:276] 0812 06:36:50.211148 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 06:36:50.211186 (+    38us) service_pool.cc:224] Handling call
0812 06:36:52.387396 (+2176210us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0812 06:36:50.214564 (+     0us) write_op.cc:183] PREPARE: starting on tablet 7065bb403e3047dd9a0853c49318ef1f
0812 06:36:50.214623 (+    59us) write_op.cc:432] Acquiring schema lock in shared mode
0812 06:36:50.214626 (+     3us) write_op.cc:435] Acquired schema lock
0812 06:36:50.214627 (+     1us) tablet.cc:662] Decoding operations
0812 06:36:50.214760 (+   133us) write_op.cc:620] Acquiring the partition lock for write op
0812 06:36:50.214765 (+     5us) write_op.cc:641] Partition lock acquired for write op
0812 06:36:50.214768 (+     3us) tablet.cc:685] Acquiring locks for 10 operations
0812 06:36:50.214824 (+    56us) tablet.cc:701] Row locks acquired
0812 06:36:50.214826 (+     2us) write_op.cc:260] PREPARE: finished
0812 06:36:50.214851 (+    25us) write_op.cc:270] Start()
0812 06:36:50.214870 (+    19us) write_op.cc:276] Timestamp: P: 1786516610214818 usec, L: 0
0812 06:36:50.214871 (+     1us) op_driver.cc:348] REPLICATION: starting
0812 06:36:50.214961 (+    90us) log.cc:844] Serialized 696 byte log entry
0812 06:36:52.359388 (+2144427us) op_driver.cc:464] REPLICATION: finished
0812 06:36:52.386986 (+ 27598us) write_op.cc:301] APPLY: starting
0812 06:36:52.386996 (+    10us) tablet.cc:1367] starting BulkCheckPresence
0812 06:36:52.387029 (+    33us) tablet.cc:1370] finished BulkCheckPresence
0812 06:36:52.387041 (+    12us) tablet.cc:1372] starting ApplyRowOperation cycle
0812 06:36:52.387203 (+   162us) tablet.cc:1383] finished ApplyRowOperation cycle
0812 06:36:52.387229 (+    26us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0812 06:36:52.387233 (+     4us) write_op.cc:312] APPLY: finished
0812 06:36:52.387286 (+    53us) log.cc:844] Serialized 77 byte log entry
0812 06:36:52.387328 (+    42us) write_op.cc:489] Releasing partition, row and schema locks
0812 06:36:52.387339 (+    11us) write_op.cc:454] Released schema lock
0812 06:36:52.387343 (+     4us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":27569,"num_ops":10,"prepare.queue_time_us":3167,"prepare.run_cpu_time_us":437,"prepare.run_wall_time_us":538,"raft.queue_time_us":26,"raft.run_cpu_time_us":4,"raft.run_wall_time_us":5,"replication_time_us":2144507}]]}
I20260812 06:36:52.374153 10118 rpcz_store.cc:276] 0812 06:36:50.205621 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 06:36:50.205661 (+    40us) service_pool.cc:224] Handling call
0812 06:36:52.374070 (+2168409us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0812 06:36:50.209245 (+     0us) write_op.cc:183] PREPARE: starting on tablet 7065bb403e3047dd9a0853c49318ef1f
0812 06:36:50.209646 (+   401us) write_op.cc:432] Acquiring schema lock in shared mode
0812 06:36:50.209649 (+     3us) write_op.cc:435] Acquired schema lock
0812 06:36:50.209651 (+     2us) tablet.cc:662] Decoding operations
0812 06:36:50.209737 (+    86us) write_op.cc:620] Acquiring the partition lock for write op
0812 06:36:50.209743 (+     6us) write_op.cc:641] Partition lock acquired for write op
0812 06:36:50.209746 (+     3us) tablet.cc:685] Acquiring locks for 10 operations
0812 06:36:50.210120 (+   374us) tablet.cc:701] Row locks acquired
0812 06:36:50.210122 (+     2us) write_op.cc:260] PREPARE: finished
0812 06:36:50.210154 (+    32us) write_op.cc:270] Start()
0812 06:36:50.210173 (+    19us) write_op.cc:276] Timestamp: P: 1786516610210120 usec, L: 0
0812 06:36:50.210175 (+     2us) op_driver.cc:348] REPLICATION: starting
0812 06:36:50.211440 (+  1265us) log.cc:844] Serialized 696 byte log entry
0812 06:36:52.358946 (+2147506us) op_driver.cc:464] REPLICATION: finished
0812 06:36:52.372823 (+ 13877us) write_op.cc:301] APPLY: starting
0812 06:36:52.372833 (+    10us) tablet.cc:1367] starting BulkCheckPresence
0812 06:36:52.372862 (+    29us) tablet.cc:1370] finished BulkCheckPresence
0812 06:36:52.373607 (+   745us) tablet.cc:1372] starting ApplyRowOperation cycle
0812 06:36:52.373812 (+   205us) tablet.cc:1383] finished ApplyRowOperation cycle
0812 06:36:52.373850 (+    38us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0812 06:36:52.373859 (+     9us) write_op.cc:312] APPLY: finished
0812 06:36:52.373930 (+    71us) log.cc:844] Serialized 77 byte log entry
0812 06:36:52.373997 (+    67us) write_op.cc:489] Releasing partition, row and schema locks
0812 06:36:52.374008 (+    11us) write_op.cc:454] Released schema lock
0812 06:36:52.374014 (+     6us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":13846,"num_ops":10,"prepare.queue_time_us":3345,"prepare.run_cpu_time_us":524,"prepare.run_wall_time_us":2306,"raft.queue_time_us":27,"raft.run_cpu_time_us":4,"raft.run_wall_time_us":5,"replication_time_us":2148755,"thread_start_us":43,"threads_started":1}]]}
I20260812 06:36:52.412241 10123 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:42054 (ReqId={client: 3067f3aef2a44cc4bb7f2ccd6acf83db, seq_no=2376, attempt_no=0}) took 2205 ms. Trace:
I20260812 06:36:52.412348 10123 rpcz_store.cc:276] 0812 06:36:50.206481 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 06:36:50.207117 (+   636us) service_pool.cc:224] Handling call
0812 06:36:52.412225 (+2205108us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0812 06:36:50.213602 (+     0us) write_op.cc:183] PREPARE: starting on tablet 7065bb403e3047dd9a0853c49318ef1f
0812 06:36:50.213663 (+    61us) write_op.cc:432] Acquiring schema lock in shared mode
0812 06:36:50.213666 (+     3us) write_op.cc:435] Acquired schema lock
0812 06:36:50.213667 (+     1us) tablet.cc:662] Decoding operations
0812 06:36:50.213744 (+    77us) write_op.cc:620] Acquiring the partition lock for write op
0812 06:36:50.213749 (+     5us) write_op.cc:641] Partition lock acquired for write op
0812 06:36:50.213752 (+     3us) tablet.cc:685] Acquiring locks for 10 operations
0812 06:36:50.213808 (+    56us) tablet.cc:701] Row locks acquired
0812 06:36:50.213809 (+     1us) write_op.cc:260] PREPARE: finished
0812 06:36:50.213834 (+    25us) write_op.cc:270] Start()
0812 06:36:50.213851 (+    17us) write_op.cc:276] Timestamp: P: 1786516610213802 usec, L: 0
0812 06:36:50.213852 (+     1us) op_driver.cc:348] REPLICATION: starting
0812 06:36:50.213945 (+    93us) log.cc:844] Serialized 686 byte log entry
0812 06:36:52.359312 (+2145367us) op_driver.cc:464] REPLICATION: finished
0812 06:36:52.380600 (+ 21288us) write_op.cc:301] APPLY: starting
0812 06:36:52.380610 (+    10us) tablet.cc:1367] starting BulkCheckPresence
0812 06:36:52.380637 (+    27us) tablet.cc:1370] finished BulkCheckPresence
0812 06:36:52.380647 (+    10us) tablet.cc:1372] starting ApplyRowOperation cycle
0812 06:36:52.380823 (+   176us) tablet.cc:1383] finished ApplyRowOperation cycle
0812 06:36:52.380849 (+    26us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0812 06:36:52.380853 (+     4us) write_op.cc:312] APPLY: finished
0812 06:36:52.380904 (+    51us) log.cc:844] Serialized 77 byte log entry
0812 06:36:52.412140 (+ 31236us) write_op.cc:489] Releasing partition, row and schema locks
0812 06:36:52.412156 (+    16us) write_op.cc:454] Released schema lock
0812 06:36:52.412161 (+     5us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":21259,"num_ops":10,"prepare.queue_time_us":6290,"prepare.run_cpu_time_us":398,"prepare.run_wall_time_us":443,"raft.queue_time_us":33,"raft.run_cpu_time_us":4,"raft.run_wall_time_us":6,"replication_time_us":2145450}]]}
I20260812 06:36:52.435352 10059 mvcc.cc:204] Tried to move back new op lower bound from 7317572035439894528 to 7317572030391631872. Current Snapshot: MvccSnapshot[applied={T|T < 7317572035437776896}]
I20260812 06:36:54.493381 10141 raft_consensus.cc:493] T 7065bb403e3047dd9a0853c49318ef1f P ec6c8d818e744cdbb6b0d15b321010e0 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 42895a9bcba64aa484f6509605035d02)
I20260812 06:36:54.493546 10141 raft_consensus.cc:515] T 7065bb403e3047dd9a0853c49318ef1f P ec6c8d818e744cdbb6b0d15b321010e0 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df4308c852240859ada53d216ab69ae" member_type: VOTER last_known_addr { host: "127.4.177.6" port: 34269 } } peers { permanent_uuid: "42895a9bcba64aa484f6509605035d02" member_type: VOTER last_known_addr { host: "127.4.177.4" port: 38181 } } peers { permanent_uuid: "222b20de5a1145c6b8a11df8374e8790" member_type: VOTER last_known_addr { host: "127.4.177.5" port: 43521 } } peers { permanent_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 40897 } } peers { permanent_uuid: "8d90d61da62549208fe117b4384a469b" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 41539 } } peers { permanent_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" member_type: VOTER last_known_addr { host: "127.4.177.7" port: 46299 } } peers { permanent_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 41057 } }
I20260812 06:36:54.494405 10141 leader_election.cc:290] T 7065bb403e3047dd9a0853c49318ef1f P ec6c8d818e744cdbb6b0d15b321010e0 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 2df4308c852240859ada53d216ab69ae (127.4.177.6:34269), 42895a9bcba64aa484f6509605035d02 (127.4.177.4:38181), 222b20de5a1145c6b8a11df8374e8790 (127.4.177.5:43521), 7962f587ccd842ad8a3f6ed9a5d7707e (127.4.177.3:40897), 8d90d61da62549208fe117b4384a469b (127.4.177.2:41539), b037d17b21f8422fa5e41b70eb4dcd08 (127.4.177.7:46299)
I20260812 06:36:54.494957  7322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" candidate_term: 19 candidate_status { last_received { term: 18 index: 2554 } } ignore_live_leader: false dest_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" is_pre_election: true
I20260812 06:36:54.495241  9421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" candidate_term: 19 candidate_status { last_received { term: 18 index: 2554 } } ignore_live_leader: false dest_uuid: "222b20de5a1145c6b8a11df8374e8790" is_pre_election: true
I20260812 06:36:54.495368  8804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" candidate_term: 19 candidate_status { last_received { term: 18 index: 2554 } } ignore_live_leader: false dest_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" is_pre_election: true
I20260812 06:36:54.495378  6236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" candidate_term: 19 candidate_status { last_received { term: 18 index: 2554 } } ignore_live_leader: false dest_uuid: "8d90d61da62549208fe117b4384a469b" is_pre_election: true
I20260812 06:36:54.495988  5945 leader_election.cc:304] T 7065bb403e3047dd9a0853c49318ef1f P ec6c8d818e744cdbb6b0d15b321010e0 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 5 responses out of 7 voters: 1 yes votes; 4 no votes. yes voters: ec6c8d818e744cdbb6b0d15b321010e0; no voters: 222b20de5a1145c6b8a11df8374e8790, 7962f587ccd842ad8a3f6ed9a5d7707e, 8d90d61da62549208fe117b4384a469b, b037d17b21f8422fa5e41b70eb4dcd08
I20260812 06:36:54.496201 10141 raft_consensus.cc:2749] T 7065bb403e3047dd9a0853c49318ef1f P ec6c8d818e744cdbb6b0d15b321010e0 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260812 06:36:54.498925  9815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" candidate_term: 19 candidate_status { last_received { term: 18 index: 2554 } } ignore_live_leader: false dest_uuid: "2df4308c852240859ada53d216ab69ae" is_pre_election: true
I20260812 06:36:54.605079 10144 raft_consensus.cc:493] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 42895a9bcba64aa484f6509605035d02)
I20260812 06:36:54.605232 10144 raft_consensus.cc:515] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df4308c852240859ada53d216ab69ae" member_type: VOTER last_known_addr { host: "127.4.177.6" port: 34269 } } peers { permanent_uuid: "42895a9bcba64aa484f6509605035d02" member_type: VOTER last_known_addr { host: "127.4.177.4" port: 38181 } } peers { permanent_uuid: "222b20de5a1145c6b8a11df8374e8790" member_type: VOTER last_known_addr { host: "127.4.177.5" port: 43521 } } peers { permanent_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 40897 } } peers { permanent_uuid: "8d90d61da62549208fe117b4384a469b" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 41539 } } peers { permanent_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" member_type: VOTER last_known_addr { host: "127.4.177.7" port: 46299 } } peers { permanent_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 41057 } }
I20260812 06:36:54.606294  9815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" candidate_term: 19 candidate_status { last_received { term: 18 index: 2588 } } ignore_live_leader: false dest_uuid: "2df4308c852240859ada53d216ab69ae" is_pre_election: true
I20260812 06:36:54.606053 10144 leader_election.cc:290] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 2df4308c852240859ada53d216ab69ae (127.4.177.6:34269), 42895a9bcba64aa484f6509605035d02 (127.4.177.4:38181), 222b20de5a1145c6b8a11df8374e8790 (127.4.177.5:43521), 7962f587ccd842ad8a3f6ed9a5d7707e (127.4.177.3:40897), 8d90d61da62549208fe117b4384a469b (127.4.177.2:41539), ec6c8d818e744cdbb6b0d15b321010e0 (127.4.177.1:41057)
I20260812 06:36:54.606568  9421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" candidate_term: 19 candidate_status { last_received { term: 18 index: 2588 } } ignore_live_leader: false dest_uuid: "222b20de5a1145c6b8a11df8374e8790" is_pre_election: true
I20260812 06:36:54.606786  6236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" candidate_term: 19 candidate_status { last_received { term: 18 index: 2588 } } ignore_live_leader: false dest_uuid: "8d90d61da62549208fe117b4384a469b" is_pre_election: true
I20260812 06:36:54.606832  6046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" candidate_term: 19 candidate_status { last_received { term: 18 index: 2588 } } ignore_live_leader: false dest_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" is_pre_election: true
I20260812 06:36:54.607036  6046 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P ec6c8d818e744cdbb6b0d15b321010e0 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b037d17b21f8422fa5e41b70eb4dcd08 in term 18.
I20260812 06:36:54.607466  8804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" candidate_term: 19 candidate_status { last_received { term: 18 index: 2588 } } ignore_live_leader: false dest_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" is_pre_election: true
I20260812 06:36:54.607867  7211 leader_election.cc:304] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 6 responses out of 7 voters: 2 yes votes; 4 no votes. yes voters: b037d17b21f8422fa5e41b70eb4dcd08, ec6c8d818e744cdbb6b0d15b321010e0; no voters: 222b20de5a1145c6b8a11df8374e8790, 2df4308c852240859ada53d216ab69ae, 7962f587ccd842ad8a3f6ed9a5d7707e, 8d90d61da62549208fe117b4384a469b
I20260812 06:36:54.608088 10144 raft_consensus.cc:2749] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260812 06:36:54.825928 10146 raft_consensus.cc:493] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 42895a9bcba64aa484f6509605035d02)
I20260812 06:36:54.826087 10146 raft_consensus.cc:515] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df4308c852240859ada53d216ab69ae" member_type: VOTER last_known_addr { host: "127.4.177.6" port: 34269 } } peers { permanent_uuid: "42895a9bcba64aa484f6509605035d02" member_type: VOTER last_known_addr { host: "127.4.177.4" port: 38181 } } peers { permanent_uuid: "222b20de5a1145c6b8a11df8374e8790" member_type: VOTER last_known_addr { host: "127.4.177.5" port: 43521 } } peers { permanent_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 40897 } } peers { permanent_uuid: "8d90d61da62549208fe117b4384a469b" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 41539 } } peers { permanent_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" member_type: VOTER last_known_addr { host: "127.4.177.7" port: 46299 } } peers { permanent_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 41057 } }
I20260812 06:36:54.826907 10146 leader_election.cc:290] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 2df4308c852240859ada53d216ab69ae (127.4.177.6:34269), 42895a9bcba64aa484f6509605035d02 (127.4.177.4:38181), 222b20de5a1145c6b8a11df8374e8790 (127.4.177.5:43521), 8d90d61da62549208fe117b4384a469b (127.4.177.2:41539), b037d17b21f8422fa5e41b70eb4dcd08 (127.4.177.7:46299), ec6c8d818e744cdbb6b0d15b321010e0 (127.4.177.1:41057)
I20260812 06:36:54.827309  9800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" candidate_term: 19 candidate_status { last_received { term: 18 index: 2669 } } ignore_live_leader: false dest_uuid: "2df4308c852240859ada53d216ab69ae" is_pre_election: true
I20260812 06:36:54.827374  7322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" candidate_term: 19 candidate_status { last_received { term: 18 index: 2669 } } ignore_live_leader: false dest_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" is_pre_election: true
I20260812 06:36:54.827401  9421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" candidate_term: 19 candidate_status { last_received { term: 18 index: 2669 } } ignore_live_leader: false dest_uuid: "222b20de5a1145c6b8a11df8374e8790" is_pre_election: true
I20260812 06:36:54.827523  7322 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7962f587ccd842ad8a3f6ed9a5d7707e in term 18.
I20260812 06:36:54.827759  6236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" candidate_term: 19 candidate_status { last_received { term: 18 index: 2669 } } ignore_live_leader: false dest_uuid: "8d90d61da62549208fe117b4384a469b" is_pre_election: true
I20260812 06:36:54.827752  6046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" candidate_term: 19 candidate_status { last_received { term: 18 index: 2669 } } ignore_live_leader: false dest_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" is_pre_election: true
I20260812 06:36:54.827927  6046 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P ec6c8d818e744cdbb6b0d15b321010e0 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7962f587ccd842ad8a3f6ed9a5d7707e in term 18.
I20260812 06:36:54.836373 10149 raft_consensus.cc:493] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 42895a9bcba64aa484f6509605035d02)
I20260812 06:36:54.836526 10149 raft_consensus.cc:515] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df4308c852240859ada53d216ab69ae" member_type: VOTER last_known_addr { host: "127.4.177.6" port: 34269 } } peers { permanent_uuid: "42895a9bcba64aa484f6509605035d02" member_type: VOTER last_known_addr { host: "127.4.177.4" port: 38181 } } peers { permanent_uuid: "222b20de5a1145c6b8a11df8374e8790" member_type: VOTER last_known_addr { host: "127.4.177.5" port: 43521 } } peers { permanent_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 40897 } } peers { permanent_uuid: "8d90d61da62549208fe117b4384a469b" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 41539 } } peers { permanent_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" member_type: VOTER last_known_addr { host: "127.4.177.7" port: 46299 } } peers { permanent_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 41057 } }
I20260812 06:36:54.837185 10150 raft_consensus.cc:493] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 42895a9bcba64aa484f6509605035d02)
I20260812 06:36:54.837327 10150 raft_consensus.cc:515] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df4308c852240859ada53d216ab69ae" member_type: VOTER last_known_addr { host: "127.4.177.6" port: 34269 } } peers { permanent_uuid: "42895a9bcba64aa484f6509605035d02" member_type: VOTER last_known_addr { host: "127.4.177.4" port: 38181 } } peers { permanent_uuid: "222b20de5a1145c6b8a11df8374e8790" member_type: VOTER last_known_addr { host: "127.4.177.5" port: 43521 } } peers { permanent_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 40897 } } peers { permanent_uuid: "8d90d61da62549208fe117b4384a469b" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 41539 } } peers { permanent_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" member_type: VOTER last_known_addr { host: "127.4.177.7" port: 46299 } } peers { permanent_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 41057 } }
I20260812 06:36:54.838837 10150 leader_election.cc:290] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 42895a9bcba64aa484f6509605035d02 (127.4.177.4:38181), 222b20de5a1145c6b8a11df8374e8790 (127.4.177.5:43521), 7962f587ccd842ad8a3f6ed9a5d7707e (127.4.177.3:40897), 8d90d61da62549208fe117b4384a469b (127.4.177.2:41539), b037d17b21f8422fa5e41b70eb4dcd08 (127.4.177.7:46299), ec6c8d818e744cdbb6b0d15b321010e0 (127.4.177.1:41057)
I20260812 06:36:54.838936 10149 leader_election.cc:290] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 2df4308c852240859ada53d216ab69ae (127.4.177.6:34269), 42895a9bcba64aa484f6509605035d02 (127.4.177.4:38181), 7962f587ccd842ad8a3f6ed9a5d7707e (127.4.177.3:40897), 8d90d61da62549208fe117b4384a469b (127.4.177.2:41539), b037d17b21f8422fa5e41b70eb4dcd08 (127.4.177.7:46299), ec6c8d818e744cdbb6b0d15b321010e0 (127.4.177.1:41057)
I20260812 06:36:54.844678  9421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "2df4308c852240859ada53d216ab69ae" candidate_term: 19 candidate_status { last_received { term: 18 index: 2676 } } ignore_live_leader: false dest_uuid: "222b20de5a1145c6b8a11df8374e8790" is_pre_election: true
I20260812 06:36:54.844846  9421 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2df4308c852240859ada53d216ab69ae in term 18.
I20260812 06:36:54.845191  9800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "222b20de5a1145c6b8a11df8374e8790" candidate_term: 19 candidate_status { last_received { term: 18 index: 2676 } } ignore_live_leader: false dest_uuid: "2df4308c852240859ada53d216ab69ae" is_pre_election: true
I20260812 06:36:54.845319  9800 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 222b20de5a1145c6b8a11df8374e8790 in term 18.
I20260812 06:36:54.848446  7322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "222b20de5a1145c6b8a11df8374e8790" candidate_term: 19 candidate_status { last_received { term: 18 index: 2676 } } ignore_live_leader: false dest_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" is_pre_election: true
I20260812 06:36:54.848619  7322 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 222b20de5a1145c6b8a11df8374e8790 in term 18.
I20260812 06:36:54.849360  8804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "2df4308c852240859ada53d216ab69ae" candidate_term: 19 candidate_status { last_received { term: 18 index: 2676 } } ignore_live_leader: false dest_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" is_pre_election: true
I20260812 06:36:54.849737  7322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "2df4308c852240859ada53d216ab69ae" candidate_term: 19 candidate_status { last_received { term: 18 index: 2676 } } ignore_live_leader: false dest_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" is_pre_election: true
I20260812 06:36:54.849874  7322 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2df4308c852240859ada53d216ab69ae in term 18.
I20260812 06:36:54.850278  8804 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2df4308c852240859ada53d216ab69ae in term 18.
I20260812 06:36:54.850349  6236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "222b20de5a1145c6b8a11df8374e8790" candidate_term: 19 candidate_status { last_received { term: 18 index: 2676 } } ignore_live_leader: false dest_uuid: "8d90d61da62549208fe117b4384a469b" is_pre_election: true
I20260812 06:36:54.850711  9714 leader_election.cc:304] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 7 voters: 4 yes votes; 0 no votes. yes voters: 222b20de5a1145c6b8a11df8374e8790, 2df4308c852240859ada53d216ab69ae, 7962f587ccd842ad8a3f6ed9a5d7707e, b037d17b21f8422fa5e41b70eb4dcd08; no voters: 
I20260812 06:36:54.850986 10150 raft_consensus.cc:2804] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260812 06:36:54.851058 10150 raft_consensus.cc:493] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [term 18 FOLLOWER]: Starting leader election (detected failure of leader 42895a9bcba64aa484f6509605035d02)
I20260812 06:36:54.851099 10150 raft_consensus.cc:3060] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [term 18 FOLLOWER]: Advancing to term 19
I20260812 06:36:54.851935  6046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "2df4308c852240859ada53d216ab69ae" candidate_term: 19 candidate_status { last_received { term: 18 index: 2676 } } ignore_live_leader: false dest_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" is_pre_election: true
I20260812 06:36:54.852077  6046 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P ec6c8d818e744cdbb6b0d15b321010e0 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2df4308c852240859ada53d216ab69ae in term 18.
I20260812 06:36:54.852344 10150 raft_consensus.cc:515] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df4308c852240859ada53d216ab69ae" member_type: VOTER last_known_addr { host: "127.4.177.6" port: 34269 } } peers { permanent_uuid: "42895a9bcba64aa484f6509605035d02" member_type: VOTER last_known_addr { host: "127.4.177.4" port: 38181 } } peers { permanent_uuid: "222b20de5a1145c6b8a11df8374e8790" member_type: VOTER last_known_addr { host: "127.4.177.5" port: 43521 } } peers { permanent_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 40897 } } peers { permanent_uuid: "8d90d61da62549208fe117b4384a469b" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 41539 } } peers { permanent_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" member_type: VOTER last_known_addr { host: "127.4.177.7" port: 46299 } } peers { permanent_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 41057 } }
I20260812 06:36:54.852439  8805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "222b20de5a1145c6b8a11df8374e8790" candidate_term: 19 candidate_status { last_received { term: 18 index: 2676 } } ignore_live_leader: false dest_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" is_pre_election: true
I20260812 06:36:54.852556  8805 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 222b20de5a1145c6b8a11df8374e8790 in term 18.
I20260812 06:36:54.852972 10150 leader_election.cc:290] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [CANDIDATE]: Term 19 election: Requested vote from peers 42895a9bcba64aa484f6509605035d02 (127.4.177.4:38181), 222b20de5a1145c6b8a11df8374e8790 (127.4.177.5:43521), 7962f587ccd842ad8a3f6ed9a5d7707e (127.4.177.3:40897), 8d90d61da62549208fe117b4384a469b (127.4.177.2:41539), b037d17b21f8422fa5e41b70eb4dcd08 (127.4.177.7:46299), ec6c8d818e744cdbb6b0d15b321010e0 (127.4.177.1:41057)
I20260812 06:36:54.852948  9315 leader_election.cc:304] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 5 responses out of 7 voters: 4 yes votes; 1 no votes. yes voters: 222b20de5a1145c6b8a11df8374e8790, 2df4308c852240859ada53d216ab69ae, 7962f587ccd842ad8a3f6ed9a5d7707e, b037d17b21f8422fa5e41b70eb4dcd08; no voters: 8d90d61da62549208fe117b4384a469b
I20260812 06:36:54.853222 10149 raft_consensus.cc:2804] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260812 06:36:54.853289 10149 raft_consensus.cc:493] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 42895a9bcba64aa484f6509605035d02)
I20260812 06:36:54.853334 10149 raft_consensus.cc:3060] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [term 18 FOLLOWER]: Advancing to term 19
I20260812 06:36:54.854035  9421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "2df4308c852240859ada53d216ab69ae" candidate_term: 19 candidate_status { last_received { term: 18 index: 2676 } } ignore_live_leader: false dest_uuid: "222b20de5a1145c6b8a11df8374e8790"
I20260812 06:36:54.854120  7322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "2df4308c852240859ada53d216ab69ae" candidate_term: 19 candidate_status { last_received { term: 18 index: 2676 } } ignore_live_leader: false dest_uuid: "b037d17b21f8422fa5e41b70eb4dcd08"
I20260812 06:36:54.854207  7322 raft_consensus.cc:3060] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08 [term 18 FOLLOWER]: Advancing to term 19
I20260812 06:36:54.854447  8805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "2df4308c852240859ada53d216ab69ae" candidate_term: 19 candidate_status { last_received { term: 18 index: 2676 } } ignore_live_leader: false dest_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e"
I20260812 06:36:54.854570  8805 raft_consensus.cc:3060] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [term 18 FOLLOWER]: Advancing to term 19
I20260812 06:36:54.854569  6236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "2df4308c852240859ada53d216ab69ae" candidate_term: 19 candidate_status { last_received { term: 18 index: 2676 } } ignore_live_leader: false dest_uuid: "8d90d61da62549208fe117b4384a469b" is_pre_election: true
I20260812 06:36:54.854631 10149 raft_consensus.cc:515] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df4308c852240859ada53d216ab69ae" member_type: VOTER last_known_addr { host: "127.4.177.6" port: 34269 } } peers { permanent_uuid: "42895a9bcba64aa484f6509605035d02" member_type: VOTER last_known_addr { host: "127.4.177.4" port: 38181 } } peers { permanent_uuid: "222b20de5a1145c6b8a11df8374e8790" member_type: VOTER last_known_addr { host: "127.4.177.5" port: 43521 } } peers { permanent_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 40897 } } peers { permanent_uuid: "8d90d61da62549208fe117b4384a469b" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 41539 } } peers { permanent_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" member_type: VOTER last_known_addr { host: "127.4.177.7" port: 46299 } } peers { permanent_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 41057 } }
I20260812 06:36:54.854839  6238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "2df4308c852240859ada53d216ab69ae" candidate_term: 19 candidate_status { last_received { term: 18 index: 2676 } } ignore_live_leader: false dest_uuid: "8d90d61da62549208fe117b4384a469b"
I20260812 06:36:54.855268 10149 leader_election.cc:290] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [CANDIDATE]: Term 19 election: Requested vote from peers 2df4308c852240859ada53d216ab69ae (127.4.177.6:34269), 42895a9bcba64aa484f6509605035d02 (127.4.177.4:38181), 7962f587ccd842ad8a3f6ed9a5d7707e (127.4.177.3:40897), 8d90d61da62549208fe117b4384a469b (127.4.177.2:41539), b037d17b21f8422fa5e41b70eb4dcd08 (127.4.177.7:46299), ec6c8d818e744cdbb6b0d15b321010e0 (127.4.177.1:41057)
I20260812 06:36:54.855273  7322 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2df4308c852240859ada53d216ab69ae in term 19.
I20260812 06:36:54.855522  9421 raft_consensus.cc:2393] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 2df4308c852240859ada53d216ab69ae in current term 19: Already voted for candidate 222b20de5a1145c6b8a11df8374e8790 in this term.
I20260812 06:36:54.855633  6046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "2df4308c852240859ada53d216ab69ae" candidate_term: 19 candidate_status { last_received { term: 18 index: 2676 } } ignore_live_leader: false dest_uuid: "ec6c8d818e744cdbb6b0d15b321010e0"
I20260812 06:36:54.855767  6046 raft_consensus.cc:3060] T 7065bb403e3047dd9a0853c49318ef1f P ec6c8d818e744cdbb6b0d15b321010e0 [term 18 FOLLOWER]: Advancing to term 19
I20260812 06:36:54.855752  8805 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2df4308c852240859ada53d216ab69ae in term 19.
I20260812 06:36:54.856439  9800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "222b20de5a1145c6b8a11df8374e8790" candidate_term: 19 candidate_status { last_received { term: 18 index: 2676 } } ignore_live_leader: false dest_uuid: "2df4308c852240859ada53d216ab69ae"
I20260812 06:36:54.856638  9800 raft_consensus.cc:2393] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 222b20de5a1145c6b8a11df8374e8790 in current term 19: Already voted for candidate 2df4308c852240859ada53d216ab69ae in this term.
I20260812 06:36:54.856616  8805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "222b20de5a1145c6b8a11df8374e8790" candidate_term: 19 candidate_status { last_received { term: 18 index: 2676 } } ignore_live_leader: false dest_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e"
I20260812 06:36:54.856763  8805 raft_consensus.cc:2393] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 222b20de5a1145c6b8a11df8374e8790 in current term 19: Already voted for candidate 2df4308c852240859ada53d216ab69ae in this term.
I20260812 06:36:54.856875  6046 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P ec6c8d818e744cdbb6b0d15b321010e0 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2df4308c852240859ada53d216ab69ae in term 19.
I20260812 06:36:54.856956  6238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "222b20de5a1145c6b8a11df8374e8790" candidate_term: 19 candidate_status { last_received { term: 18 index: 2676 } } ignore_live_leader: false dest_uuid: "8d90d61da62549208fe117b4384a469b"
I20260812 06:36:54.857079  7322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "222b20de5a1145c6b8a11df8374e8790" candidate_term: 19 candidate_status { last_received { term: 18 index: 2676 } } ignore_live_leader: false dest_uuid: "b037d17b21f8422fa5e41b70eb4dcd08"
I20260812 06:36:54.857205  7322 raft_consensus.cc:2393] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 222b20de5a1145c6b8a11df8374e8790 in current term 19: Already voted for candidate 2df4308c852240859ada53d216ab69ae in this term.
I20260812 06:36:54.857175  9714 leader_election.cc:304] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 6 responses out of 7 voters: 4 yes votes; 2 no votes. yes voters: 2df4308c852240859ada53d216ab69ae, 7962f587ccd842ad8a3f6ed9a5d7707e, b037d17b21f8422fa5e41b70eb4dcd08, ec6c8d818e744cdbb6b0d15b321010e0; no voters: 222b20de5a1145c6b8a11df8374e8790, 8d90d61da62549208fe117b4384a469b
I20260812 06:36:54.857257  6048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "222b20de5a1145c6b8a11df8374e8790" candidate_term: 19 candidate_status { last_received { term: 18 index: 2676 } } ignore_live_leader: false dest_uuid: "ec6c8d818e744cdbb6b0d15b321010e0"
I20260812 06:36:54.857378 10150 raft_consensus.cc:2804] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [term 19 FOLLOWER]: Leader election won for term 19
I20260812 06:36:54.857402  6048 raft_consensus.cc:2393] T 7065bb403e3047dd9a0853c49318ef1f P ec6c8d818e744cdbb6b0d15b321010e0 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 222b20de5a1145c6b8a11df8374e8790 in current term 19: Already voted for candidate 2df4308c852240859ada53d216ab69ae in this term.
I20260812 06:36:54.857662  6046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "222b20de5a1145c6b8a11df8374e8790" candidate_term: 19 candidate_status { last_received { term: 18 index: 2676 } } ignore_live_leader: false dest_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" is_pre_election: true
I20260812 06:36:54.857744 10150 raft_consensus.cc:697] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [term 19 LEADER]: Becoming Leader. State: Replica: 2df4308c852240859ada53d216ab69ae, State: Running, Role: LEADER
I20260812 06:36:54.857808  6046 raft_consensus.cc:2393] T 7065bb403e3047dd9a0853c49318ef1f P ec6c8d818e744cdbb6b0d15b321010e0 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 222b20de5a1145c6b8a11df8374e8790 in current term 19: Already voted for candidate 2df4308c852240859ada53d216ab69ae in this term.
I20260812 06:36:54.857790  9315 leader_election.cc:304] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 5 responses out of 7 voters: 1 yes votes; 4 no votes. yes voters: 222b20de5a1145c6b8a11df8374e8790; no voters: 2df4308c852240859ada53d216ab69ae, 7962f587ccd842ad8a3f6ed9a5d7707e, 8d90d61da62549208fe117b4384a469b, b037d17b21f8422fa5e41b70eb4dcd08
I20260812 06:36:54.857983 10149 raft_consensus.cc:2749] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260812 06:36:54.858110 10150 consensus_queue.cc:237] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2668, Committed index: 2668, Last appended: 18.2676, Last appended by leader: 2676, Current term: 19, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df4308c852240859ada53d216ab69ae" member_type: VOTER last_known_addr { host: "127.4.177.6" port: 34269 } } peers { permanent_uuid: "42895a9bcba64aa484f6509605035d02" member_type: VOTER last_known_addr { host: "127.4.177.4" port: 38181 } } peers { permanent_uuid: "222b20de5a1145c6b8a11df8374e8790" member_type: VOTER last_known_addr { host: "127.4.177.5" port: 43521 } } peers { permanent_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 40897 } } peers { permanent_uuid: "8d90d61da62549208fe117b4384a469b" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 41539 } } peers { permanent_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" member_type: VOTER last_known_addr { host: "127.4.177.7" port: 46299 } } peers { permanent_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 41057 } }
I20260812 06:36:54.861261  5810 catalog_manager.cc:5719] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae reported cstate change: term changed from 18 to 19, leader changed from 42895a9bcba64aa484f6509605035d02 (127.4.177.4) to 2df4308c852240859ada53d216ab69ae (127.4.177.6). New cstate: current_term: 19 leader_uuid: "2df4308c852240859ada53d216ab69ae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df4308c852240859ada53d216ab69ae" member_type: VOTER last_known_addr { host: "127.4.177.6" port: 34269 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "42895a9bcba64aa484f6509605035d02" member_type: VOTER last_known_addr { host: "127.4.177.4" port: 38181 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "222b20de5a1145c6b8a11df8374e8790" member_type: VOTER last_known_addr { host: "127.4.177.5" port: 43521 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 40897 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8d90d61da62549208fe117b4384a469b" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 41539 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" member_type: VOTER last_known_addr { host: "127.4.177.7" port: 46299 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 41057 } health_report { overall_health: UNKNOWN } } }
W20260812 06:36:54.913862  6597 connection.cc:256] RPC call timeout handler was delayed by -1.021714576985687s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1s for a response
I20260812 06:36:54.914248  6692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" candidate_term: 19 candidate_status { last_received { term: 18 index: 2669 } } ignore_live_leader: false dest_uuid: "42895a9bcba64aa484f6509605035d02" is_pre_election: true
I20260812 06:36:54.914727  6672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" candidate_term: 19 candidate_status { last_received { term: 18 index: 2588 } } ignore_live_leader: false dest_uuid: "42895a9bcba64aa484f6509605035d02" is_pre_election: true
I20260812 06:36:54.915221  9421 raft_consensus.cc:1240] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [term 19 FOLLOWER]: Rejecting Update request from peer 42895a9bcba64aa484f6509605035d02 for earlier term 18. Current term is 19. Ops: []
I20260812 06:36:54.918375  8698 leader_election.cc:304] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 7 responses out of 7 voters: 3 yes votes; 4 no votes. yes voters: 7962f587ccd842ad8a3f6ed9a5d7707e, b037d17b21f8422fa5e41b70eb4dcd08, ec6c8d818e744cdbb6b0d15b321010e0; no voters: 222b20de5a1145c6b8a11df8374e8790, 2df4308c852240859ada53d216ab69ae, 42895a9bcba64aa484f6509605035d02, 8d90d61da62549208fe117b4384a469b
I20260812 06:36:54.918716 10146 raft_consensus.cc:2749] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260812 06:36:54.920111  7322 raft_consensus.cc:1240] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08 [term 19 FOLLOWER]: Rejecting Update request from peer 42895a9bcba64aa484f6509605035d02 for earlier term 18. Current term is 19. Ops: [18.2589-18.2676]
I20260812 06:36:54.921003  8805 raft_consensus.cc:1240] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [term 19 FOLLOWER]: Rejecting Update request from peer 42895a9bcba64aa484f6509605035d02 for earlier term 18. Current term is 19. Ops: [18.2670-18.2676]
I20260812 06:36:54.921715 10139 consensus_queue.cc:1059] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 40897 }, Status: INVALID_TERM, Last received: 18.2669, Next index: 2670, Last known committed idx: 2667, Time since last communication: 0.000s
I20260812 06:36:54.921877 10139 raft_consensus.cc:3055] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [term 18 LEADER]: Stepping down as leader of term 18
I20260812 06:36:54.921948 10139 raft_consensus.cc:740] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 42895a9bcba64aa484f6509605035d02, State: Running, Role: LEADER
I20260812 06:36:54.921968  6046 raft_consensus.cc:1240] T 7065bb403e3047dd9a0853c49318ef1f P ec6c8d818e744cdbb6b0d15b321010e0 [term 19 FOLLOWER]: Rejecting Update request from peer 42895a9bcba64aa484f6509605035d02 for earlier term 18. Current term is 19. Ops: [18.2555-18.2676]
I20260812 06:36:54.922104 10139 consensus_queue.cc:260] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2669, Committed index: 2669, Last appended: 18.2676, Last appended by leader: 2676, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df4308c852240859ada53d216ab69ae" member_type: VOTER last_known_addr { host: "127.4.177.6" port: 34269 } } peers { permanent_uuid: "42895a9bcba64aa484f6509605035d02" member_type: VOTER last_known_addr { host: "127.4.177.4" port: 38181 } } peers { permanent_uuid: "222b20de5a1145c6b8a11df8374e8790" member_type: VOTER last_known_addr { host: "127.4.177.5" port: 43521 } } peers { permanent_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 40897 } } peers { permanent_uuid: "8d90d61da62549208fe117b4384a469b" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 41539 } } peers { permanent_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" member_type: VOTER last_known_addr { host: "127.4.177.7" port: 46299 } } peers { permanent_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 41057 } }
I20260812 06:36:54.922376 10139 raft_consensus.cc:3060] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [term 18 FOLLOWER]: Advancing to term 19
I20260812 06:36:54.922398  9800 raft_consensus.cc:1240] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [term 19 LEADER]: Rejecting Update request from peer 42895a9bcba64aa484f6509605035d02 for earlier term 18. Current term is 19. Ops: []
I20260812 06:36:54.922801  6703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" candidate_term: 19 candidate_status { last_received { term: 18 index: 2554 } } ignore_live_leader: false dest_uuid: "42895a9bcba64aa484f6509605035d02" is_pre_election: true
I20260812 06:36:54.923743  6703 raft_consensus.cc:2410] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ec6c8d818e744cdbb6b0d15b321010e0 for term 19 because replica has last-logged OpId of term: 18 index: 2676, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 2554.
I20260812 06:36:54.925463  6686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "222b20de5a1145c6b8a11df8374e8790" candidate_term: 19 candidate_status { last_received { term: 18 index: 2676 } } ignore_live_leader: false dest_uuid: "42895a9bcba64aa484f6509605035d02"
I20260812 06:36:54.926625  6686 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 222b20de5a1145c6b8a11df8374e8790 in term 19.
I20260812 06:36:54.930713  6681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "2df4308c852240859ada53d216ab69ae" candidate_term: 19 candidate_status { last_received { term: 18 index: 2676 } } ignore_live_leader: false dest_uuid: "42895a9bcba64aa484f6509605035d02"
I20260812 06:36:54.930850  6681 raft_consensus.cc:2393] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 2df4308c852240859ada53d216ab69ae in current term 19: Already voted for candidate 222b20de5a1145c6b8a11df8374e8790 in this term.
I20260812 06:36:54.932227  6703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "222b20de5a1145c6b8a11df8374e8790" candidate_term: 19 candidate_status { last_received { term: 18 index: 2676 } } ignore_live_leader: false dest_uuid: "42895a9bcba64aa484f6509605035d02" is_pre_election: true
I20260812 06:36:54.932264 10200 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:42408 (ReqId={client: 3067f3aef2a44cc4bb7f2ccd6acf83db, seq_no=2654, attempt_no=0}) took 1616 ms. Trace:
I20260812 06:36:54.932343  6703 raft_consensus.cc:2376] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [term 19 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 222b20de5a1145c6b8a11df8374e8790 in term 19. Re-sending same reply.
I20260812 06:36:54.932934  6686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "2df4308c852240859ada53d216ab69ae" candidate_term: 19 candidate_status { last_received { term: 18 index: 2676 } } ignore_live_leader: false dest_uuid: "42895a9bcba64aa484f6509605035d02" is_pre_election: true
I20260812 06:36:54.933066  6686 raft_consensus.cc:2393] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2df4308c852240859ada53d216ab69ae in current term 19: Already voted for candidate 222b20de5a1145c6b8a11df8374e8790 in this term.
I20260812 06:36:54.932325 10200 rpcz_store.cc:276] 0812 06:36:53.315587 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 06:36:53.317037 (+  1450us) service_pool.cc:224] Handling call
0812 06:36:54.932237 (+1615200us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0812 06:36:53.320284 (+     0us) write_op.cc:183] PREPARE: starting on tablet 7065bb403e3047dd9a0853c49318ef1f
0812 06:36:53.320362 (+    78us) write_op.cc:432] Acquiring schema lock in shared mode
0812 06:36:53.320365 (+     3us) write_op.cc:435] Acquired schema lock
0812 06:36:53.320366 (+     1us) tablet.cc:662] Decoding operations
0812 06:36:53.320452 (+    86us) write_op.cc:620] Acquiring the partition lock for write op
0812 06:36:53.320458 (+     6us) write_op.cc:641] Partition lock acquired for write op
0812 06:36:53.320462 (+     4us) tablet.cc:685] Acquiring locks for 10 operations
0812 06:36:53.320529 (+    67us) tablet.cc:701] Row locks acquired
0812 06:36:53.320531 (+     2us) write_op.cc:260] PREPARE: finished
0812 06:36:53.320560 (+    29us) write_op.cc:270] Start()
0812 06:36:53.320946 (+   386us) write_op.cc:276] Timestamp: P: 1786516613320531 usec, L: 0
0812 06:36:53.320949 (+     3us) op_driver.cc:348] REPLICATION: starting
0812 06:36:53.321071 (+   122us) log.cc:844] Serialized 696 byte log entry
0812 06:36:54.916045 (+1594974us) op_driver.cc:464] REPLICATION: finished
0812 06:36:54.931638 (+ 15593us) write_op.cc:301] APPLY: starting
0812 06:36:54.931653 (+    15us) tablet.cc:1367] starting BulkCheckPresence
0812 06:36:54.931684 (+    31us) tablet.cc:1370] finished BulkCheckPresence
0812 06:36:54.931696 (+    12us) tablet.cc:1372] starting ApplyRowOperation cycle
0812 06:36:54.931887 (+   191us) tablet.cc:1383] finished ApplyRowOperation cycle
0812 06:36:54.931914 (+    27us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0812 06:36:54.931919 (+     5us) write_op.cc:312] APPLY: finished
0812 06:36:54.931978 (+    59us) log.cc:844] Serialized 77 byte log entry
0812 06:36:54.932155 (+   177us) write_op.cc:489] Releasing partition, row and schema locks
0812 06:36:54.932171 (+    16us) write_op.cc:454] Released schema lock
0812 06:36:54.932176 (+     5us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":15540,"num_ops":10,"prepare.queue_time_us":2992,"prepare.run_cpu_time_us":560,"prepare.run_wall_time_us":3016,"raft.queue_time_us":155,"raft.run_cpu_time_us":4,"raft.run_wall_time_us":4,"replication_time_us":1595065,"spinlock_wait_cycles":66944,"thread_start_us":172,"threads_started":2,"wal-append.queue_time_us":7865}]]}
W20260812 06:36:54.942245  6623 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:42408: Illegal state: replica 42895a9bcba64aa484f6509605035d02 is not leader of this config: current role FOLLOWER
W20260812 06:36:54.945616  8742 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:48246: Illegal state: replica 7962f587ccd842ad8a3f6ed9a5d7707e is not leader of this config: current role FOLLOWER
W20260812 06:36:54.952926  6179 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:42054: Illegal state: replica 8d90d61da62549208fe117b4384a469b is not leader of this config: current role FOLLOWER
W20260812 06:36:54.961994  7249 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:58454: Illegal state: replica b037d17b21f8422fa5e41b70eb4dcd08 is not leader of this config: current role FOLLOWER
W20260812 06:36:54.969684  5984 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:42334: Illegal state: replica ec6c8d818e744cdbb6b0d15b321010e0 is not leader of this config: current role FOLLOWER
I20260812 06:36:54.985271  7322 raft_consensus.cc:1275] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08 [term 19 FOLLOWER]: Refusing update from remote peer 2df4308c852240859ada53d216ab69ae: Log matching property violated. Preceding OpId in replica: term: 18 index: 2588. Preceding OpId from leader: term: 19 index: 2678. (index mismatch)
I20260812 06:36:54.985271  6238 raft_consensus.cc:3060] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b [term 18 FOLLOWER]: Advancing to term 19
I20260812 06:36:54.985570  8805 raft_consensus.cc:1275] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [term 19 FOLLOWER]: Refusing update from remote peer 2df4308c852240859ada53d216ab69ae: Log matching property violated. Preceding OpId in replica: term: 18 index: 2669. Preceding OpId from leader: term: 19 index: 2678. (index mismatch)
I20260812 06:36:54.985663  6046 raft_consensus.cc:1275] T 7065bb403e3047dd9a0853c49318ef1f P ec6c8d818e744cdbb6b0d15b321010e0 [term 19 FOLLOWER]: Refusing update from remote peer 2df4308c852240859ada53d216ab69ae: Log matching property violated. Preceding OpId in replica: term: 18 index: 2554. Preceding OpId from leader: term: 19 index: 2678. (index mismatch)
I20260812 06:36:54.985968  6686 raft_consensus.cc:1275] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [term 19 FOLLOWER]: Refusing update from remote peer 2df4308c852240859ada53d216ab69ae: Log matching property violated. Preceding OpId in replica: term: 18 index: 2676. Preceding OpId from leader: term: 19 index: 2678. (index mismatch)
I20260812 06:36:54.986217 10220 consensus_queue.cc:1048] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" member_type: VOTER last_known_addr { host: "127.4.177.7" port: 46299 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2677, Last known committed idx: 2587, Time since last communication: 0.000s
I20260812 06:36:54.986620  6238 raft_consensus.cc:1275] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b [term 19 FOLLOWER]: Refusing update from remote peer 2df4308c852240859ada53d216ab69ae: Log matching property violated. Preceding OpId in replica: term: 18 index: 2676. Preceding OpId from leader: term: 19 index: 2678. (index mismatch)
I20260812 06:36:54.986680 10150 consensus_queue.cc:1048] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 40897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2677, Last known committed idx: 2667, Time since last communication: 0.000s
I20260812 06:36:54.986863  9421 raft_consensus.cc:1275] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [term 19 FOLLOWER]: Refusing update from remote peer 2df4308c852240859ada53d216ab69ae: Log matching property violated. Preceding OpId in replica: term: 18 index: 2676. Preceding OpId from leader: term: 19 index: 2678. (index mismatch)
I20260812 06:36:54.987031 10150 consensus_queue.cc:1048] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "42895a9bcba64aa484f6509605035d02" member_type: VOTER last_known_addr { host: "127.4.177.4" port: 38181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2677, Last known committed idx: 2669, Time since last communication: 0.000s
I20260812 06:36:54.987249 10150 consensus_queue.cc:1048] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 41057 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2677, Last known committed idx: 2550, Time since last communication: 0.000s
I20260812 06:36:54.987419 10189 consensus_queue.cc:1048] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "8d90d61da62549208fe117b4384a469b" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 41539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2677, Last known committed idx: 2669, Time since last communication: 0.000s
I20260812 06:36:54.987975 10189 consensus_queue.cc:1048] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "222b20de5a1145c6b8a11df8374e8790" member_type: VOTER last_known_addr { host: "127.4.177.5" port: 43521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2677, Last known committed idx: 2668, Time since last communication: 0.000s
I20260812 06:36:55.009137 10200 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:42408 (ReqId={client: 3067f3aef2a44cc4bb7f2ccd6acf83db, seq_no=2656, attempt_no=0}) took 1691 ms. Trace:
I20260812 06:36:55.009244 10200 rpcz_store.cc:276] 0812 06:36:53.317880 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 06:36:53.318301 (+   421us) service_pool.cc:224] Handling call
0812 06:36:55.009122 (+1690821us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0812 06:36:53.323331 (+     0us) write_op.cc:183] PREPARE: starting on tablet 7065bb403e3047dd9a0853c49318ef1f
0812 06:36:53.323651 (+   320us) write_op.cc:432] Acquiring schema lock in shared mode
0812 06:36:53.323655 (+     4us) write_op.cc:435] Acquired schema lock
0812 06:36:53.323657 (+     2us) tablet.cc:662] Decoding operations
0812 06:36:53.323752 (+    95us) write_op.cc:620] Acquiring the partition lock for write op
0812 06:36:53.323759 (+     7us) write_op.cc:641] Partition lock acquired for write op
0812 06:36:53.323763 (+     4us) tablet.cc:685] Acquiring locks for 10 operations
0812 06:36:53.323826 (+    63us) tablet.cc:701] Row locks acquired
0812 06:36:53.323828 (+     2us) write_op.cc:260] PREPARE: finished
0812 06:36:53.323860 (+    32us) write_op.cc:270] Start()
0812 06:36:53.323880 (+    20us) write_op.cc:276] Timestamp: P: 1786516613323829 usec, L: 0
0812 06:36:53.323882 (+     2us) op_driver.cc:348] REPLICATION: starting
0812 06:36:53.324006 (+   124us) log.cc:844] Serialized 696 byte log entry
0812 06:36:55.007813 (+1683807us) op_driver.cc:464] REPLICATION: finished
0812 06:36:55.008650 (+   837us) write_op.cc:301] APPLY: starting
0812 06:36:55.008664 (+    14us) tablet.cc:1367] starting BulkCheckPresence
0812 06:36:55.008696 (+    32us) tablet.cc:1370] finished BulkCheckPresence
0812 06:36:55.008708 (+    12us) tablet.cc:1372] starting ApplyRowOperation cycle
0812 06:36:55.008912 (+   204us) tablet.cc:1383] finished ApplyRowOperation cycle
0812 06:36:55.008944 (+    32us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0812 06:36:55.008949 (+     5us) write_op.cc:312] APPLY: finished
0812 06:36:55.009005 (+    56us) log.cc:844] Serialized 77 byte log entry
0812 06:36:55.009045 (+    40us) write_op.cc:489] Releasing partition, row and schema locks
0812 06:36:55.009059 (+    14us) write_op.cc:454] Released schema lock
0812 06:36:55.009064 (+     5us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":793,"num_ops":10,"prepare.queue_time_us":4785,"prepare.run_cpu_time_us":483,"prepare.run_wall_time_us":942,"replication_time_us":1683908}]]}
I20260812 06:36:55.010048 10200 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:42408 (ReqId={client: 3067f3aef2a44cc4bb7f2ccd6acf83db, seq_no=2657, attempt_no=0}) took 1691 ms. Trace:
I20260812 06:36:55.010120 10200 rpcz_store.cc:276] 0812 06:36:53.318603 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 06:36:53.318644 (+    41us) service_pool.cc:224] Handling call
0812 06:36:55.010038 (+1691394us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0812 06:36:53.324304 (+     0us) write_op.cc:183] PREPARE: starting on tablet 7065bb403e3047dd9a0853c49318ef1f
0812 06:36:53.324384 (+    80us) write_op.cc:432] Acquiring schema lock in shared mode
0812 06:36:53.324389 (+     5us) write_op.cc:435] Acquired schema lock
0812 06:36:53.324392 (+     3us) tablet.cc:662] Decoding operations
0812 06:36:53.324488 (+    96us) write_op.cc:620] Acquiring the partition lock for write op
0812 06:36:53.324494 (+     6us) write_op.cc:641] Partition lock acquired for write op
0812 06:36:53.324497 (+     3us) tablet.cc:685] Acquiring locks for 10 operations
0812 06:36:53.324569 (+    72us) tablet.cc:701] Row locks acquired
0812 06:36:53.324570 (+     1us) write_op.cc:260] PREPARE: finished
0812 06:36:53.324633 (+    63us) write_op.cc:270] Start()
0812 06:36:53.324654 (+    21us) write_op.cc:276] Timestamp: P: 1786516613324601 usec, L: 0
0812 06:36:53.324655 (+     1us) op_driver.cc:348] REPLICATION: starting
0812 06:36:53.325475 (+   820us) log.cc:844] Serialized 696 byte log entry
0812 06:36:55.007872 (+1682397us) op_driver.cc:464] REPLICATION: finished
0812 06:36:55.009632 (+  1760us) write_op.cc:301] APPLY: starting
0812 06:36:55.009642 (+    10us) tablet.cc:1367] starting BulkCheckPresence
0812 06:36:55.009672 (+    30us) tablet.cc:1370] finished BulkCheckPresence
0812 06:36:55.009684 (+    12us) tablet.cc:1372] starting ApplyRowOperation cycle
0812 06:36:55.009856 (+   172us) tablet.cc:1383] finished ApplyRowOperation cycle
0812 06:36:55.009882 (+    26us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0812 06:36:55.009886 (+     4us) write_op.cc:312] APPLY: finished
0812 06:36:55.009931 (+    45us) log.cc:844] Serialized 77 byte log entry
0812 06:36:55.009964 (+    33us) write_op.cc:489] Releasing partition, row and schema locks
0812 06:36:55.009979 (+    15us) write_op.cc:454] Released schema lock
0812 06:36:55.009983 (+     4us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":1733,"num_ops":10,"prepare.queue_time_us":5433,"prepare.run_cpu_time_us":512,"prepare.run_wall_time_us":1411,"replication_time_us":1683203,"thread_start_us":65,"threads_started":1}]]}
I20260812 06:36:55.010933 10200 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:42408 (ReqId={client: 3067f3aef2a44cc4bb7f2ccd6acf83db, seq_no=2659, attempt_no=0}) took 1686 ms. Trace:
I20260812 06:36:55.011000 10200 rpcz_store.cc:276] 0812 06:36:53.324652 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 06:36:53.324690 (+    38us) service_pool.cc:224] Handling call
0812 06:36:55.010923 (+1686233us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0812 06:36:53.325739 (+     0us) write_op.cc:183] PREPARE: starting on tablet 7065bb403e3047dd9a0853c49318ef1f
0812 06:36:53.326610 (+   871us) write_op.cc:432] Acquiring schema lock in shared mode
0812 06:36:53.326613 (+     3us) write_op.cc:435] Acquired schema lock
0812 06:36:53.326615 (+     2us) tablet.cc:662] Decoding operations
0812 06:36:53.326726 (+   111us) write_op.cc:620] Acquiring the partition lock for write op
0812 06:36:53.326733 (+     7us) write_op.cc:641] Partition lock acquired for write op
0812 06:36:53.326736 (+     3us) tablet.cc:685] Acquiring locks for 10 operations
0812 06:36:53.326794 (+    58us) tablet.cc:701] Row locks acquired
0812 06:36:53.326795 (+     1us) write_op.cc:260] PREPARE: finished
0812 06:36:53.326847 (+    52us) write_op.cc:270] Start()
0812 06:36:53.326867 (+    20us) write_op.cc:276] Timestamp: P: 1786516613326816 usec, L: 0
0812 06:36:53.326870 (+     3us) op_driver.cc:348] REPLICATION: starting
0812 06:36:53.327008 (+   138us) log.cc:844] Serialized 686 byte log entry
0812 06:36:55.007990 (+1680982us) op_driver.cc:464] REPLICATION: finished
0812 06:36:55.010507 (+  2517us) write_op.cc:301] APPLY: starting
0812 06:36:55.010516 (+     9us) tablet.cc:1367] starting BulkCheckPresence
0812 06:36:55.010546 (+    30us) tablet.cc:1370] finished BulkCheckPresence
0812 06:36:55.010562 (+    16us) tablet.cc:1372] starting ApplyRowOperation cycle
0812 06:36:55.010743 (+   181us) tablet.cc:1383] finished ApplyRowOperation cycle
0812 06:36:55.010768 (+    25us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0812 06:36:55.010773 (+     5us) write_op.cc:312] APPLY: finished
0812 06:36:55.010828 (+    55us) log.cc:844] Serialized 77 byte log entry
0812 06:36:55.010861 (+    33us) write_op.cc:489] Releasing partition, row and schema locks
0812 06:36:55.010872 (+    11us) write_op.cc:454] Released schema lock
0812 06:36:55.010877 (+     5us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"spinlock_wait_cycles":768,"child_traces":[["op",{"apply.queue_time_us":2486,"num_ops":10,"prepare.queue_time_us":773,"prepare.run_cpu_time_us":532,"prepare.run_wall_time_us":1555,"replication_time_us":1681102,"thread_start_us":58,"threads_started":1}]]}
I20260812 06:36:55.011812 10200 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:42408 (ReqId={client: 3067f3aef2a44cc4bb7f2ccd6acf83db, seq_no=2660, attempt_no=0}) took 1687 ms. Trace:
I20260812 06:36:55.011878 10200 rpcz_store.cc:276] 0812 06:36:53.324730 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 06:36:53.324773 (+    43us) service_pool.cc:224] Handling call
0812 06:36:55.011803 (+1687030us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0812 06:36:53.327329 (+     0us) write_op.cc:183] PREPARE: starting on tablet 7065bb403e3047dd9a0853c49318ef1f
0812 06:36:53.327425 (+    96us) write_op.cc:432] Acquiring schema lock in shared mode
0812 06:36:53.327431 (+     6us) write_op.cc:435] Acquired schema lock
0812 06:36:53.327434 (+     3us) tablet.cc:662] Decoding operations
0812 06:36:53.327578 (+   144us) write_op.cc:620] Acquiring the partition lock for write op
0812 06:36:53.327586 (+     8us) write_op.cc:641] Partition lock acquired for write op
0812 06:36:53.327592 (+     6us) tablet.cc:685] Acquiring locks for 10 operations
0812 06:36:53.327699 (+   107us) tablet.cc:701] Row locks acquired
0812 06:36:53.327702 (+     3us) write_op.cc:260] PREPARE: finished
0812 06:36:53.327732 (+    30us) write_op.cc:270] Start()
0812 06:36:53.327757 (+    25us) write_op.cc:276] Timestamp: P: 1786516613327703 usec, L: 0
0812 06:36:53.327761 (+     4us) op_driver.cc:348] REPLICATION: starting
0812 06:36:53.327880 (+   119us) log.cc:844] Serialized 696 byte log entry
0812 06:36:55.008096 (+1680216us) op_driver.cc:464] REPLICATION: finished
0812 06:36:55.011350 (+  3254us) write_op.cc:301] APPLY: starting
0812 06:36:55.011359 (+     9us) tablet.cc:1367] starting BulkCheckPresence
0812 06:36:55.011404 (+    45us) tablet.cc:1370] finished BulkCheckPresence
0812 06:36:55.011420 (+    16us) tablet.cc:1372] starting ApplyRowOperation cycle
0812 06:36:55.011629 (+   209us) tablet.cc:1383] finished ApplyRowOperation cycle
0812 06:36:55.011655 (+    26us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0812 06:36:55.011659 (+     4us) write_op.cc:312] APPLY: finished
0812 06:36:55.011706 (+    47us) log.cc:844] Serialized 77 byte log entry
0812 06:36:55.011739 (+    33us) write_op.cc:489] Releasing partition, row and schema locks
0812 06:36:55.011751 (+    12us) write_op.cc:454] Released schema lock
0812 06:36:55.011756 (+     5us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"spinlock_wait_cycles":19072,"child_traces":[["op",{"apply.queue_time_us":3226,"mutex_wait_us":29,"num_ops":10,"prepare.queue_time_us":2359,"prepare.run_cpu_time_us":613,"prepare.run_wall_time_us":613,"replication_time_us":1680319,"thread_start_us":97,"threads_started":1}]]}
I20260812 06:36:55.012724 10200 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:42408 (ReqId={client: 3067f3aef2a44cc4bb7f2ccd6acf83db, seq_no=2658, attempt_no=0}) took 1688 ms. Trace:
I20260812 06:36:55.012789 10200 rpcz_store.cc:276] 0812 06:36:53.324535 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 06:36:53.324842 (+   307us) service_pool.cc:224] Handling call
0812 06:36:55.012715 (+1687873us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0812 06:36:53.327980 (+     0us) write_op.cc:183] PREPARE: starting on tablet 7065bb403e3047dd9a0853c49318ef1f
0812 06:36:53.328048 (+    68us) write_op.cc:432] Acquiring schema lock in shared mode
0812 06:36:53.328051 (+     3us) write_op.cc:435] Acquired schema lock
0812 06:36:53.328052 (+     1us) tablet.cc:662] Decoding operations
0812 06:36:53.328133 (+    81us) write_op.cc:620] Acquiring the partition lock for write op
0812 06:36:53.328139 (+     6us) write_op.cc:641] Partition lock acquired for write op
0812 06:36:53.328142 (+     3us) tablet.cc:685] Acquiring locks for 10 operations
0812 06:36:53.328202 (+    60us) tablet.cc:701] Row locks acquired
0812 06:36:53.328203 (+     1us) write_op.cc:260] PREPARE: finished
0812 06:36:53.328228 (+    25us) write_op.cc:270] Start()
0812 06:36:53.328245 (+    17us) write_op.cc:276] Timestamp: P: 1786516613328201 usec, L: 0
0812 06:36:53.328247 (+     2us) op_driver.cc:348] REPLICATION: starting
0812 06:36:53.328357 (+   110us) log.cc:844] Serialized 696 byte log entry
0812 06:36:55.008249 (+1679892us) op_driver.cc:464] REPLICATION: finished
0812 06:36:55.012214 (+  3965us) write_op.cc:301] APPLY: starting
0812 06:36:55.012224 (+    10us) tablet.cc:1367] starting BulkCheckPresence
0812 06:36:55.012250 (+    26us) tablet.cc:1370] finished BulkCheckPresence
0812 06:36:55.012268 (+    18us) tablet.cc:1372] starting ApplyRowOperation cycle
0812 06:36:55.012540 (+   272us) tablet.cc:1383] finished ApplyRowOperation cycle
0812 06:36:55.012566 (+    26us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0812 06:36:55.012571 (+     5us) write_op.cc:312] APPLY: finished
0812 06:36:55.012622 (+    51us) log.cc:844] Serialized 77 byte log entry
0812 06:36:55.012654 (+    32us) write_op.cc:489] Releasing partition, row and schema locks
0812 06:36:55.012666 (+    12us) write_op.cc:454] Released schema lock
0812 06:36:55.012670 (+     4us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":3931,"num_ops":10,"prepare.queue_time_us":2870,"prepare.run_cpu_time_us":431,"prepare.run_wall_time_us":430,"replication_time_us":1679984}]]}
I20260812 06:36:55.013499 10200 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:42408 (ReqId={client: 3067f3aef2a44cc4bb7f2ccd6acf83db, seq_no=2661, attempt_no=0}) took 1688 ms. Trace:
I20260812 06:36:55.013562 10200 rpcz_store.cc:276] 0812 06:36:53.324807 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 06:36:53.325153 (+   346us) service_pool.cc:224] Handling call
0812 06:36:55.013490 (+1688337us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0812 06:36:53.328439 (+     0us) write_op.cc:183] PREPARE: starting on tablet 7065bb403e3047dd9a0853c49318ef1f
0812 06:36:53.328508 (+    69us) write_op.cc:432] Acquiring schema lock in shared mode
0812 06:36:53.328511 (+     3us) write_op.cc:435] Acquired schema lock
0812 06:36:53.328512 (+     1us) tablet.cc:662] Decoding operations
0812 06:36:53.328596 (+    84us) write_op.cc:620] Acquiring the partition lock for write op
0812 06:36:53.328602 (+     6us) write_op.cc:641] Partition lock acquired for write op
0812 06:36:53.328604 (+     2us) tablet.cc:685] Acquiring locks for 10 operations
0812 06:36:53.328662 (+    58us) tablet.cc:701] Row locks acquired
0812 06:36:53.328664 (+     2us) write_op.cc:260] PREPARE: finished
0812 06:36:53.328688 (+    24us) write_op.cc:270] Start()
0812 06:36:53.328704 (+    16us) write_op.cc:276] Timestamp: P: 1786516613328660 usec, L: 0
0812 06:36:53.328706 (+     2us) op_driver.cc:348] REPLICATION: starting
0812 06:36:53.328804 (+    98us) log.cc:844] Serialized 686 byte log entry
0812 06:36:55.008282 (+1679478us) op_driver.cc:464] REPLICATION: finished
0812 06:36:55.013086 (+  4804us) write_op.cc:301] APPLY: starting
0812 06:36:55.013095 (+     9us) tablet.cc:1367] starting BulkCheckPresence
0812 06:36:55.013120 (+    25us) tablet.cc:1370] finished BulkCheckPresence
0812 06:36:55.013131 (+    11us) tablet.cc:1372] starting ApplyRowOperation cycle
0812 06:36:55.013305 (+   174us) tablet.cc:1383] finished ApplyRowOperation cycle
0812 06:36:55.013330 (+    25us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0812 06:36:55.013335 (+     5us) write_op.cc:312] APPLY: finished
0812 06:36:55.013394 (+    59us) log.cc:844] Serialized 77 byte log entry
0812 06:36:55.013427 (+    33us) write_op.cc:489] Releasing partition, row and schema locks
0812 06:36:55.013437 (+    10us) write_op.cc:454] Released schema lock
0812 06:36:55.013441 (+     4us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":4780,"num_ops":10,"prepare.queue_time_us":3067,"prepare.run_cpu_time_us":417,"prepare.run_wall_time_us":417,"replication_time_us":1679566}]]}
I20260812 06:36:55.025615 10200 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:42408 (ReqId={client: 3067f3aef2a44cc4bb7f2ccd6acf83db, seq_no=2662, attempt_no=0}) took 1692 ms. Trace:
I20260812 06:36:55.025717 10200 rpcz_store.cc:276] 0812 06:36:53.333397 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 06:36:53.333465 (+    68us) service_pool.cc:224] Handling call
0812 06:36:55.025600 (+1692135us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0812 06:36:53.333796 (+     0us) write_op.cc:183] PREPARE: starting on tablet 7065bb403e3047dd9a0853c49318ef1f
0812 06:36:53.333876 (+    80us) write_op.cc:432] Acquiring schema lock in shared mode
0812 06:36:53.333879 (+     3us) write_op.cc:435] Acquired schema lock
0812 06:36:53.333880 (+     1us) tablet.cc:662] Decoding operations
0812 06:36:53.334203 (+   323us) write_op.cc:620] Acquiring the partition lock for write op
0812 06:36:53.334239 (+    36us) write_op.cc:641] Partition lock acquired for write op
0812 06:36:53.334243 (+     4us) tablet.cc:685] Acquiring locks for 10 operations
0812 06:36:53.334307 (+    64us) tablet.cc:701] Row locks acquired
0812 06:36:53.334309 (+     2us) write_op.cc:260] PREPARE: finished
0812 06:36:53.334337 (+    28us) write_op.cc:270] Start()
0812 06:36:53.334356 (+    19us) write_op.cc:276] Timestamp: P: 1786516613334308 usec, L: 0
0812 06:36:53.334358 (+     2us) op_driver.cc:348] REPLICATION: starting
0812 06:36:53.334477 (+   119us) log.cc:844] Serialized 696 byte log entry
0812 06:36:55.008308 (+1673831us) op_driver.cc:464] REPLICATION: finished
0812 06:36:55.013866 (+  5558us) write_op.cc:301] APPLY: starting
0812 06:36:55.013875 (+     9us) tablet.cc:1367] starting BulkCheckPresence
0812 06:36:55.013902 (+    27us) tablet.cc:1370] finished BulkCheckPresence
0812 06:36:55.013912 (+    10us) tablet.cc:1372] starting ApplyRowOperation cycle
0812 06:36:55.014086 (+   174us) tablet.cc:1383] finished ApplyRowOperation cycle
0812 06:36:55.014111 (+    25us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0812 06:36:55.014115 (+     4us) write_op.cc:312] APPLY: finished
0812 06:36:55.014164 (+    49us) log.cc:844] Serialized 77 byte log entry
0812 06:36:55.014195 (+    31us) write_op.cc:489] Releasing partition, row and schema locks
0812 06:36:55.014206 (+    11us) write_op.cc:454] Released schema lock
0812 06:36:55.025526 (+ 11320us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":5533,"num_ops":10,"prepare.queue_time_us":42,"prepare.run_cpu_time_us":511,"prepare.run_wall_time_us":1146,"replication_time_us":1673939}]]}
I20260812 06:36:56.980430 10313 raft_consensus.cc:493] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 2df4308c852240859ada53d216ab69ae)
I20260812 06:36:56.980584 10313 raft_consensus.cc:515] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df4308c852240859ada53d216ab69ae" member_type: VOTER last_known_addr { host: "127.4.177.6" port: 34269 } } peers { permanent_uuid: "42895a9bcba64aa484f6509605035d02" member_type: VOTER last_known_addr { host: "127.4.177.4" port: 38181 } } peers { permanent_uuid: "222b20de5a1145c6b8a11df8374e8790" member_type: VOTER last_known_addr { host: "127.4.177.5" port: 43521 } } peers { permanent_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 40897 } } peers { permanent_uuid: "8d90d61da62549208fe117b4384a469b" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 41539 } } peers { permanent_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" member_type: VOTER last_known_addr { host: "127.4.177.7" port: 46299 } } peers { permanent_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 41057 } }
I20260812 06:36:56.981424 10313 leader_election.cc:290] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 2df4308c852240859ada53d216ab69ae (127.4.177.6:34269), 42895a9bcba64aa484f6509605035d02 (127.4.177.4:38181), 222b20de5a1145c6b8a11df8374e8790 (127.4.177.5:43521), 7962f587ccd842ad8a3f6ed9a5d7707e (127.4.177.3:40897), 8d90d61da62549208fe117b4384a469b (127.4.177.2:41539), ec6c8d818e744cdbb6b0d15b321010e0 (127.4.177.1:41057)
I20260812 06:36:56.981992  9421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" candidate_term: 20 candidate_status { last_received { term: 19 index: 2790 } } ignore_live_leader: false dest_uuid: "222b20de5a1145c6b8a11df8374e8790" is_pre_election: true
I20260812 06:36:56.981988  6703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" candidate_term: 20 candidate_status { last_received { term: 19 index: 2790 } } ignore_live_leader: false dest_uuid: "42895a9bcba64aa484f6509605035d02" is_pre_election: true
I20260812 06:36:56.982108  8805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" candidate_term: 20 candidate_status { last_received { term: 19 index: 2790 } } ignore_live_leader: false dest_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" is_pre_election: true
I20260812 06:36:56.982399  6238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" candidate_term: 20 candidate_status { last_received { term: 19 index: 2790 } } ignore_live_leader: false dest_uuid: "8d90d61da62549208fe117b4384a469b" is_pre_election: true
I20260812 06:36:56.982420  6046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" candidate_term: 20 candidate_status { last_received { term: 19 index: 2790 } } ignore_live_leader: false dest_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" is_pre_election: true
I20260812 06:36:56.982769  7211 leader_election.cc:304] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 5 responses out of 7 voters: 1 yes votes; 4 no votes. yes voters: b037d17b21f8422fa5e41b70eb4dcd08; no voters: 222b20de5a1145c6b8a11df8374e8790, 42895a9bcba64aa484f6509605035d02, 7962f587ccd842ad8a3f6ed9a5d7707e, ec6c8d818e744cdbb6b0d15b321010e0
I20260812 06:36:56.982981 10313 raft_consensus.cc:2749] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260812 06:36:57.219309 10317 raft_consensus.cc:493] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 2df4308c852240859ada53d216ab69ae)
I20260812 06:36:57.219565 10317 raft_consensus.cc:515] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df4308c852240859ada53d216ab69ae" member_type: VOTER last_known_addr { host: "127.4.177.6" port: 34269 } } peers { permanent_uuid: "42895a9bcba64aa484f6509605035d02" member_type: VOTER last_known_addr { host: "127.4.177.4" port: 38181 } } peers { permanent_uuid: "222b20de5a1145c6b8a11df8374e8790" member_type: VOTER last_known_addr { host: "127.4.177.5" port: 43521 } } peers { permanent_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 40897 } } peers { permanent_uuid: "8d90d61da62549208fe117b4384a469b" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 41539 } } peers { permanent_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" member_type: VOTER last_known_addr { host: "127.4.177.7" port: 46299 } } peers { permanent_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 41057 } }
I20260812 06:36:57.219729 10318 raft_consensus.cc:493] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 2df4308c852240859ada53d216ab69ae)
I20260812 06:36:57.219905 10318 raft_consensus.cc:515] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df4308c852240859ada53d216ab69ae" member_type: VOTER last_known_addr { host: "127.4.177.6" port: 34269 } } peers { permanent_uuid: "42895a9bcba64aa484f6509605035d02" member_type: VOTER last_known_addr { host: "127.4.177.4" port: 38181 } } peers { permanent_uuid: "222b20de5a1145c6b8a11df8374e8790" member_type: VOTER last_known_addr { host: "127.4.177.5" port: 43521 } } peers { permanent_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 40897 } } peers { permanent_uuid: "8d90d61da62549208fe117b4384a469b" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 41539 } } peers { permanent_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" member_type: VOTER last_known_addr { host: "127.4.177.7" port: 46299 } } peers { permanent_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 41057 } }
I20260812 06:36:57.220395 10317 leader_election.cc:290] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 2df4308c852240859ada53d216ab69ae (127.4.177.6:34269), 42895a9bcba64aa484f6509605035d02 (127.4.177.4:38181), 7962f587ccd842ad8a3f6ed9a5d7707e (127.4.177.3:40897), 8d90d61da62549208fe117b4384a469b (127.4.177.2:41539), b037d17b21f8422fa5e41b70eb4dcd08 (127.4.177.7:46299), ec6c8d818e744cdbb6b0d15b321010e0 (127.4.177.1:41057)
I20260812 06:36:57.220715 10318 leader_election.cc:290] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 2df4308c852240859ada53d216ab69ae (127.4.177.6:34269), 222b20de5a1145c6b8a11df8374e8790 (127.4.177.5:43521), 7962f587ccd842ad8a3f6ed9a5d7707e (127.4.177.3:40897), 8d90d61da62549208fe117b4384a469b (127.4.177.2:41539), b037d17b21f8422fa5e41b70eb4dcd08 (127.4.177.7:46299), ec6c8d818e744cdbb6b0d15b321010e0 (127.4.177.1:41057)
I20260812 06:36:57.220918  7322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "222b20de5a1145c6b8a11df8374e8790" candidate_term: 20 candidate_status { last_received { term: 19 index: 2887 } } ignore_live_leader: false dest_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" is_pre_election: true
I20260812 06:36:57.221027  6703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "222b20de5a1145c6b8a11df8374e8790" candidate_term: 20 candidate_status { last_received { term: 19 index: 2887 } } ignore_live_leader: false dest_uuid: "42895a9bcba64aa484f6509605035d02" is_pre_election: true
I20260812 06:36:57.221068  7322 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 222b20de5a1145c6b8a11df8374e8790 in term 19.
I20260812 06:36:57.221095  7320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "42895a9bcba64aa484f6509605035d02" candidate_term: 20 candidate_status { last_received { term: 19 index: 2887 } } ignore_live_leader: false dest_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" is_pre_election: true
I20260812 06:36:57.221151  6703 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 222b20de5a1145c6b8a11df8374e8790 in term 19.
I20260812 06:36:57.221204  7320 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 42895a9bcba64aa484f6509605035d02 in term 19.
I20260812 06:36:57.221179  6046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "222b20de5a1145c6b8a11df8374e8790" candidate_term: 20 candidate_status { last_received { term: 19 index: 2887 } } ignore_live_leader: false dest_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" is_pre_election: true
I20260812 06:36:57.221424  6238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "222b20de5a1145c6b8a11df8374e8790" candidate_term: 20 candidate_status { last_received { term: 19 index: 2887 } } ignore_live_leader: false dest_uuid: "8d90d61da62549208fe117b4384a469b" is_pre_election: true
I20260812 06:36:57.221601  8805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "222b20de5a1145c6b8a11df8374e8790" candidate_term: 20 candidate_status { last_received { term: 19 index: 2887 } } ignore_live_leader: false dest_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" is_pre_election: true
I20260812 06:36:57.222126  8805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "42895a9bcba64aa484f6509605035d02" candidate_term: 20 candidate_status { last_received { term: 19 index: 2887 } } ignore_live_leader: false dest_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" is_pre_election: true
I20260812 06:36:57.222126  6238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "42895a9bcba64aa484f6509605035d02" candidate_term: 20 candidate_status { last_received { term: 19 index: 2887 } } ignore_live_leader: false dest_uuid: "8d90d61da62549208fe117b4384a469b" is_pre_election: true
I20260812 06:36:57.222203  6046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "42895a9bcba64aa484f6509605035d02" candidate_term: 20 candidate_status { last_received { term: 19 index: 2887 } } ignore_live_leader: false dest_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" is_pre_election: true
I20260812 06:36:57.222204  9421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "42895a9bcba64aa484f6509605035d02" candidate_term: 20 candidate_status { last_received { term: 19 index: 2887 } } ignore_live_leader: false dest_uuid: "222b20de5a1145c6b8a11df8374e8790" is_pre_election: true
I20260812 06:36:57.222342  9421 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 42895a9bcba64aa484f6509605035d02 in term 19.
I20260812 06:36:57.551993 10319 raft_consensus.cc:493] T 7065bb403e3047dd9a0853c49318ef1f P ec6c8d818e744cdbb6b0d15b321010e0 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 2df4308c852240859ada53d216ab69ae)
I20260812 06:36:57.552147 10319 raft_consensus.cc:515] T 7065bb403e3047dd9a0853c49318ef1f P ec6c8d818e744cdbb6b0d15b321010e0 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df4308c852240859ada53d216ab69ae" member_type: VOTER last_known_addr { host: "127.4.177.6" port: 34269 } } peers { permanent_uuid: "42895a9bcba64aa484f6509605035d02" member_type: VOTER last_known_addr { host: "127.4.177.4" port: 38181 } } peers { permanent_uuid: "222b20de5a1145c6b8a11df8374e8790" member_type: VOTER last_known_addr { host: "127.4.177.5" port: 43521 } } peers { permanent_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 40897 } } peers { permanent_uuid: "8d90d61da62549208fe117b4384a469b" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 41539 } } peers { permanent_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" member_type: VOTER last_known_addr { host: "127.4.177.7" port: 46299 } } peers { permanent_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 41057 } }
I20260812 06:36:57.552969 10319 leader_election.cc:290] T 7065bb403e3047dd9a0853c49318ef1f P ec6c8d818e744cdbb6b0d15b321010e0 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 2df4308c852240859ada53d216ab69ae (127.4.177.6:34269), 42895a9bcba64aa484f6509605035d02 (127.4.177.4:38181), 222b20de5a1145c6b8a11df8374e8790 (127.4.177.5:43521), 7962f587ccd842ad8a3f6ed9a5d7707e (127.4.177.3:40897), 8d90d61da62549208fe117b4384a469b (127.4.177.2:41539), b037d17b21f8422fa5e41b70eb4dcd08 (127.4.177.7:46299)
I20260812 06:36:57.553527  7320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" candidate_term: 20 candidate_status { last_received { term: 19 index: 2756 } } ignore_live_leader: false dest_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" is_pre_election: true
I20260812 06:36:57.553642  9421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" candidate_term: 20 candidate_status { last_received { term: 19 index: 2756 } } ignore_live_leader: false dest_uuid: "222b20de5a1145c6b8a11df8374e8790" is_pre_election: true
I20260812 06:36:57.553718  7320 raft_consensus.cc:2410] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ec6c8d818e744cdbb6b0d15b321010e0 for term 20 because replica has last-logged OpId of term: 19 index: 2790, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 2756.
I20260812 06:36:57.553714  6703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" candidate_term: 20 candidate_status { last_received { term: 19 index: 2756 } } ignore_live_leader: false dest_uuid: "42895a9bcba64aa484f6509605035d02" is_pre_election: true
I20260812 06:36:57.553718  8805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" candidate_term: 20 candidate_status { last_received { term: 19 index: 2756 } } ignore_live_leader: false dest_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" is_pre_election: true
I20260812 06:36:57.553848  9421 raft_consensus.cc:2410] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ec6c8d818e744cdbb6b0d15b321010e0 for term 20 because replica has last-logged OpId of term: 19 index: 2887, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 2756.
I20260812 06:36:57.553870  6703 raft_consensus.cc:2410] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ec6c8d818e744cdbb6b0d15b321010e0 for term 20 because replica has last-logged OpId of term: 19 index: 2887, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 2756.
I20260812 06:36:57.554096  6238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" candidate_term: 20 candidate_status { last_received { term: 19 index: 2756 } } ignore_live_leader: false dest_uuid: "8d90d61da62549208fe117b4384a469b" is_pre_election: true
I20260812 06:36:57.554352  5945 leader_election.cc:304] T 7065bb403e3047dd9a0853c49318ef1f P ec6c8d818e744cdbb6b0d15b321010e0 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 5 responses out of 7 voters: 1 yes votes; 4 no votes. yes voters: ec6c8d818e744cdbb6b0d15b321010e0; no voters: 222b20de5a1145c6b8a11df8374e8790, 42895a9bcba64aa484f6509605035d02, 7962f587ccd842ad8a3f6ed9a5d7707e, b037d17b21f8422fa5e41b70eb4dcd08
I20260812 06:36:57.554592 10319 raft_consensus.cc:2749] T 7065bb403e3047dd9a0853c49318ef1f P ec6c8d818e744cdbb6b0d15b321010e0 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260812 06:36:57.556622 10320 raft_consensus.cc:493] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 2df4308c852240859ada53d216ab69ae)
I20260812 06:36:57.556751 10320 raft_consensus.cc:515] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df4308c852240859ada53d216ab69ae" member_type: VOTER last_known_addr { host: "127.4.177.6" port: 34269 } } peers { permanent_uuid: "42895a9bcba64aa484f6509605035d02" member_type: VOTER last_known_addr { host: "127.4.177.4" port: 38181 } } peers { permanent_uuid: "222b20de5a1145c6b8a11df8374e8790" member_type: VOTER last_known_addr { host: "127.4.177.5" port: 43521 } } peers { permanent_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 40897 } } peers { permanent_uuid: "8d90d61da62549208fe117b4384a469b" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 41539 } } peers { permanent_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" member_type: VOTER last_known_addr { host: "127.4.177.7" port: 46299 } } peers { permanent_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 41057 } }
I20260812 06:36:57.557437 10320 leader_election.cc:290] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 2df4308c852240859ada53d216ab69ae (127.4.177.6:34269), 42895a9bcba64aa484f6509605035d02 (127.4.177.4:38181), 222b20de5a1145c6b8a11df8374e8790 (127.4.177.5:43521), 7962f587ccd842ad8a3f6ed9a5d7707e (127.4.177.3:40897), b037d17b21f8422fa5e41b70eb4dcd08 (127.4.177.7:46299), ec6c8d818e744cdbb6b0d15b321010e0 (127.4.177.1:41057)
I20260812 06:36:57.557829  6703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "8d90d61da62549208fe117b4384a469b" candidate_term: 20 candidate_status { last_received { term: 19 index: 2830 } } ignore_live_leader: false dest_uuid: "42895a9bcba64aa484f6509605035d02" is_pre_election: true
I20260812 06:36:57.557850  9421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "8d90d61da62549208fe117b4384a469b" candidate_term: 20 candidate_status { last_received { term: 19 index: 2830 } } ignore_live_leader: false dest_uuid: "222b20de5a1145c6b8a11df8374e8790" is_pre_election: true
I20260812 06:36:57.558005  9421 raft_consensus.cc:2410] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8d90d61da62549208fe117b4384a469b for term 20 because replica has last-logged OpId of term: 19 index: 2887, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 2830.
I20260812 06:36:57.557928  6046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "8d90d61da62549208fe117b4384a469b" candidate_term: 20 candidate_status { last_received { term: 19 index: 2830 } } ignore_live_leader: false dest_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" is_pre_election: true
I20260812 06:36:57.558009  6703 raft_consensus.cc:2410] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8d90d61da62549208fe117b4384a469b for term 20 because replica has last-logged OpId of term: 19 index: 2887, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 2830.
I20260812 06:36:57.557976  8805 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "8d90d61da62549208fe117b4384a469b" candidate_term: 20 candidate_status { last_received { term: 19 index: 2830 } } ignore_live_leader: false dest_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" is_pre_election: true
I20260812 06:36:57.558074  6046 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P ec6c8d818e744cdbb6b0d15b321010e0 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8d90d61da62549208fe117b4384a469b in term 19.
I20260812 06:36:57.558452  7320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "8d90d61da62549208fe117b4384a469b" candidate_term: 20 candidate_status { last_received { term: 19 index: 2830 } } ignore_live_leader: false dest_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" is_pre_election: true
I20260812 06:36:57.558640  7320 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8d90d61da62549208fe117b4384a469b in term 19.
I20260812 06:36:57.567696 10322 raft_consensus.cc:493] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 2df4308c852240859ada53d216ab69ae)
I20260812 06:36:57.567855 10322 raft_consensus.cc:515] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df4308c852240859ada53d216ab69ae" member_type: VOTER last_known_addr { host: "127.4.177.6" port: 34269 } } peers { permanent_uuid: "42895a9bcba64aa484f6509605035d02" member_type: VOTER last_known_addr { host: "127.4.177.4" port: 38181 } } peers { permanent_uuid: "222b20de5a1145c6b8a11df8374e8790" member_type: VOTER last_known_addr { host: "127.4.177.5" port: 43521 } } peers { permanent_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 40897 } } peers { permanent_uuid: "8d90d61da62549208fe117b4384a469b" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 41539 } } peers { permanent_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" member_type: VOTER last_known_addr { host: "127.4.177.7" port: 46299 } } peers { permanent_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 41057 } }
I20260812 06:36:57.568647 10322 leader_election.cc:290] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 2df4308c852240859ada53d216ab69ae (127.4.177.6:34269), 42895a9bcba64aa484f6509605035d02 (127.4.177.4:38181), 222b20de5a1145c6b8a11df8374e8790 (127.4.177.5:43521), 8d90d61da62549208fe117b4384a469b (127.4.177.2:41539), b037d17b21f8422fa5e41b70eb4dcd08 (127.4.177.7:46299), ec6c8d818e744cdbb6b0d15b321010e0 (127.4.177.1:41057)
I20260812 06:36:57.568944  6703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" candidate_term: 20 candidate_status { last_received { term: 19 index: 2880 } } ignore_live_leader: false dest_uuid: "42895a9bcba64aa484f6509605035d02" is_pre_election: true
I20260812 06:36:57.569123  6703 raft_consensus.cc:2410] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7962f587ccd842ad8a3f6ed9a5d7707e for term 20 because replica has last-logged OpId of term: 19 index: 2887, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 2880.
I20260812 06:36:57.569068  9421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" candidate_term: 20 candidate_status { last_received { term: 19 index: 2880 } } ignore_live_leader: false dest_uuid: "222b20de5a1145c6b8a11df8374e8790" is_pre_election: true
I20260812 06:36:57.569092  6046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" candidate_term: 20 candidate_status { last_received { term: 19 index: 2880 } } ignore_live_leader: false dest_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" is_pre_election: true
I20260812 06:36:57.569123  6238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" candidate_term: 20 candidate_status { last_received { term: 19 index: 2880 } } ignore_live_leader: false dest_uuid: "8d90d61da62549208fe117b4384a469b" is_pre_election: true
I20260812 06:36:57.569216  6046 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P ec6c8d818e744cdbb6b0d15b321010e0 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7962f587ccd842ad8a3f6ed9a5d7707e in term 19.
I20260812 06:36:57.569242  9421 raft_consensus.cc:2410] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7962f587ccd842ad8a3f6ed9a5d7707e for term 20 because replica has last-logged OpId of term: 19 index: 2887, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 2880.
I20260812 06:36:57.569254  6238 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7962f587ccd842ad8a3f6ed9a5d7707e in term 19.
I20260812 06:36:57.569656  7320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" candidate_term: 20 candidate_status { last_received { term: 19 index: 2880 } } ignore_live_leader: false dest_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" is_pre_election: true
I20260812 06:36:57.569782  7320 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7962f587ccd842ad8a3f6ed9a5d7707e in term 19.
I20260812 06:36:57.570087  8695 leader_election.cc:304] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 6 responses out of 7 voters: 4 yes votes; 2 no votes. yes voters: 7962f587ccd842ad8a3f6ed9a5d7707e, 8d90d61da62549208fe117b4384a469b, b037d17b21f8422fa5e41b70eb4dcd08, ec6c8d818e744cdbb6b0d15b321010e0; no voters: 222b20de5a1145c6b8a11df8374e8790, 42895a9bcba64aa484f6509605035d02
I20260812 06:36:57.570304 10322 raft_consensus.cc:2804] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260812 06:36:57.570369 10322 raft_consensus.cc:493] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [term 19 FOLLOWER]: Starting leader election (detected failure of leader 2df4308c852240859ada53d216ab69ae)
I20260812 06:36:57.570396 10322 raft_consensus.cc:3060] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [term 19 FOLLOWER]: Advancing to term 20
I20260812 06:36:57.571290 10322 raft_consensus.cc:515] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df4308c852240859ada53d216ab69ae" member_type: VOTER last_known_addr { host: "127.4.177.6" port: 34269 } } peers { permanent_uuid: "42895a9bcba64aa484f6509605035d02" member_type: VOTER last_known_addr { host: "127.4.177.4" port: 38181 } } peers { permanent_uuid: "222b20de5a1145c6b8a11df8374e8790" member_type: VOTER last_known_addr { host: "127.4.177.5" port: 43521 } } peers { permanent_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 40897 } } peers { permanent_uuid: "8d90d61da62549208fe117b4384a469b" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 41539 } } peers { permanent_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" member_type: VOTER last_known_addr { host: "127.4.177.7" port: 46299 } } peers { permanent_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 41057 } }
I20260812 06:36:57.571810 10322 leader_election.cc:290] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [CANDIDATE]: Term 20 election: Requested vote from peers 2df4308c852240859ada53d216ab69ae (127.4.177.6:34269), 42895a9bcba64aa484f6509605035d02 (127.4.177.4:38181), 222b20de5a1145c6b8a11df8374e8790 (127.4.177.5:43521), 8d90d61da62549208fe117b4384a469b (127.4.177.2:41539), b037d17b21f8422fa5e41b70eb4dcd08 (127.4.177.7:46299), ec6c8d818e744cdbb6b0d15b321010e0 (127.4.177.1:41057)
I20260812 06:36:57.572084  6703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" candidate_term: 20 candidate_status { last_received { term: 19 index: 2880 } } ignore_live_leader: false dest_uuid: "42895a9bcba64aa484f6509605035d02"
I20260812 06:36:57.572108  7320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" candidate_term: 20 candidate_status { last_received { term: 19 index: 2880 } } ignore_live_leader: false dest_uuid: "b037d17b21f8422fa5e41b70eb4dcd08"
I20260812 06:36:57.572216  7320 raft_consensus.cc:3060] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08 [term 19 FOLLOWER]: Advancing to term 20
I20260812 06:36:57.572218  6703 raft_consensus.cc:3060] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [term 19 FOLLOWER]: Advancing to term 20
I20260812 06:36:57.572357  6238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" candidate_term: 20 candidate_status { last_received { term: 19 index: 2880 } } ignore_live_leader: false dest_uuid: "8d90d61da62549208fe117b4384a469b"
I20260812 06:36:57.572366  6046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" candidate_term: 20 candidate_status { last_received { term: 19 index: 2880 } } ignore_live_leader: false dest_uuid: "ec6c8d818e744cdbb6b0d15b321010e0"
I20260812 06:36:57.572474  6238 raft_consensus.cc:3060] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b [term 19 FOLLOWER]: Advancing to term 20
I20260812 06:36:57.572489  6046 raft_consensus.cc:3060] T 7065bb403e3047dd9a0853c49318ef1f P ec6c8d818e744cdbb6b0d15b321010e0 [term 19 FOLLOWER]: Advancing to term 20
I20260812 06:36:57.573303  7320 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7962f587ccd842ad8a3f6ed9a5d7707e in term 20.
I20260812 06:36:57.573493  6703 raft_consensus.cc:2410] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 7962f587ccd842ad8a3f6ed9a5d7707e for term 20 because replica has last-logged OpId of term: 19 index: 2887, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 2880.
I20260812 06:36:57.573510  6238 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7962f587ccd842ad8a3f6ed9a5d7707e in term 20.
I20260812 06:36:57.573616  6046 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P ec6c8d818e744cdbb6b0d15b321010e0 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7962f587ccd842ad8a3f6ed9a5d7707e in term 20.
I20260812 06:36:57.573755  9421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" candidate_term: 20 candidate_status { last_received { term: 19 index: 2880 } } ignore_live_leader: false dest_uuid: "222b20de5a1145c6b8a11df8374e8790"
I20260812 06:36:57.573885  9421 raft_consensus.cc:3060] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [term 19 FOLLOWER]: Advancing to term 20
I20260812 06:36:57.573980  8698 leader_election.cc:304] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 5 responses out of 7 voters: 4 yes votes; 1 no votes. yes voters: 7962f587ccd842ad8a3f6ed9a5d7707e, 8d90d61da62549208fe117b4384a469b, b037d17b21f8422fa5e41b70eb4dcd08, ec6c8d818e744cdbb6b0d15b321010e0; no voters: 42895a9bcba64aa484f6509605035d02
I20260812 06:36:57.574241 10322 raft_consensus.cc:2804] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [term 20 FOLLOWER]: Leader election won for term 20
I20260812 06:36:57.574442 10322 raft_consensus.cc:697] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [term 20 LEADER]: Becoming Leader. State: Replica: 7962f587ccd842ad8a3f6ed9a5d7707e, State: Running, Role: LEADER
I20260812 06:36:57.574796 10322 consensus_queue.cc:237] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2878, Committed index: 2878, Last appended: 19.2880, Last appended by leader: 2880, Current term: 20, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df4308c852240859ada53d216ab69ae" member_type: VOTER last_known_addr { host: "127.4.177.6" port: 34269 } } peers { permanent_uuid: "42895a9bcba64aa484f6509605035d02" member_type: VOTER last_known_addr { host: "127.4.177.4" port: 38181 } } peers { permanent_uuid: "222b20de5a1145c6b8a11df8374e8790" member_type: VOTER last_known_addr { host: "127.4.177.5" port: 43521 } } peers { permanent_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 40897 } } peers { permanent_uuid: "8d90d61da62549208fe117b4384a469b" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 41539 } } peers { permanent_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" member_type: VOTER last_known_addr { host: "127.4.177.7" port: 46299 } } peers { permanent_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 41057 } }
I20260812 06:36:57.575075  9421 raft_consensus.cc:2410] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 7962f587ccd842ad8a3f6ed9a5d7707e for term 20 because replica has last-logged OpId of term: 19 index: 2887, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 2880.
I20260812 06:36:57.577546  5814 catalog_manager.cc:5719] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e reported cstate change: term changed from 19 to 20, leader changed from 2df4308c852240859ada53d216ab69ae (127.4.177.6) to 7962f587ccd842ad8a3f6ed9a5d7707e (127.4.177.3). New cstate: current_term: 20 leader_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df4308c852240859ada53d216ab69ae" member_type: VOTER last_known_addr { host: "127.4.177.6" port: 34269 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "42895a9bcba64aa484f6509605035d02" member_type: VOTER last_known_addr { host: "127.4.177.4" port: 38181 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "222b20de5a1145c6b8a11df8374e8790" member_type: VOTER last_known_addr { host: "127.4.177.5" port: 43521 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 40897 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8d90d61da62549208fe117b4384a469b" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 41539 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" member_type: VOTER last_known_addr { host: "127.4.177.7" port: 46299 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 41057 } health_report { overall_health: UNKNOWN } } }
W20260812 06:36:57.907155  9714 connection.cc:256] RPC call timeout handler was delayed by -1.634896761039272s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1s for a response
W20260812 06:36:57.907284  9714 connection.cc:256] RPC call timeout handler was delayed by -1.4160017690155655s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1s for a response
W20260812 06:36:57.907316  9714 connection.cc:256] RPC call timeout handler was delayed by -1.2980598399881274s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1s for a response
I20260812 06:36:57.907404  9800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "222b20de5a1145c6b8a11df8374e8790" candidate_term: 20 candidate_status { last_received { term: 19 index: 2887 } } ignore_live_leader: false dest_uuid: "2df4308c852240859ada53d216ab69ae" is_pre_election: true
W20260812 06:36:57.908378  9710 connection.cc:256] RPC call timeout handler was delayed by -1.5289283420424908s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1s for a response
I20260812 06:36:57.908613  9808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "42895a9bcba64aa484f6509605035d02" candidate_term: 20 candidate_status { last_received { term: 19 index: 2887 } } ignore_live_leader: false dest_uuid: "2df4308c852240859ada53d216ab69ae" is_pre_election: true
I20260812 06:36:57.908697  9815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" candidate_term: 20 candidate_status { last_received { term: 19 index: 2880 } } ignore_live_leader: false dest_uuid: "2df4308c852240859ada53d216ab69ae" is_pre_election: true
I20260812 06:36:57.908928  9810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "8d90d61da62549208fe117b4384a469b" candidate_term: 20 candidate_status { last_received { term: 19 index: 2830 } } ignore_live_leader: false dest_uuid: "2df4308c852240859ada53d216ab69ae" is_pre_election: true
I20260812 06:36:57.909276  9814 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" candidate_term: 20 candidate_status { last_received { term: 19 index: 2880 } } ignore_live_leader: false dest_uuid: "2df4308c852240859ada53d216ab69ae"
I20260812 06:36:57.910161 10255 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:41528 (ReqId={client: 3067f3aef2a44cc4bb7f2ccd6acf83db, seq_no=2866, attempt_no=0}) took 2207 ms. Trace:
I20260812 06:36:57.910826  9798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" candidate_term: 20 candidate_status { last_received { term: 19 index: 2756 } } ignore_live_leader: false dest_uuid: "2df4308c852240859ada53d216ab69ae" is_pre_election: true
I20260812 06:36:57.910271 10255 rpcz_store.cc:276] 0812 06:36:55.702295 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 06:36:55.702336 (+    41us) service_pool.cc:224] Handling call
0812 06:36:57.910137 (+2207801us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0812 06:36:55.703773 (+     0us) write_op.cc:183] PREPARE: starting on tablet 7065bb403e3047dd9a0853c49318ef1f
0812 06:36:55.703888 (+   115us) write_op.cc:432] Acquiring schema lock in shared mode
0812 06:36:55.703892 (+     4us) write_op.cc:435] Acquired schema lock
0812 06:36:55.703894 (+     2us) tablet.cc:662] Decoding operations
0812 06:36:55.704019 (+   125us) write_op.cc:620] Acquiring the partition lock for write op
0812 06:36:55.704026 (+     7us) write_op.cc:641] Partition lock acquired for write op
0812 06:36:55.704029 (+     3us) tablet.cc:685] Acquiring locks for 10 operations
0812 06:36:55.704095 (+    66us) tablet.cc:701] Row locks acquired
0812 06:36:55.704097 (+     2us) write_op.cc:260] PREPARE: finished
0812 06:36:55.704130 (+    33us) write_op.cc:270] Start()
0812 06:36:55.704154 (+    24us) write_op.cc:276] Timestamp: P: 1786516615704067 usec, L: 0
0812 06:36:55.704155 (+     1us) op_driver.cc:348] REPLICATION: starting
0812 06:36:55.704274 (+   119us) log.cc:844] Serialized 686 byte log entry
0812 06:36:57.907974 (+2203700us) op_driver.cc:464] REPLICATION: finished
0812 06:36:57.909523 (+  1549us) write_op.cc:301] APPLY: starting
0812 06:36:57.909542 (+    19us) tablet.cc:1367] starting BulkCheckPresence
0812 06:36:57.909590 (+    48us) tablet.cc:1370] finished BulkCheckPresence
0812 06:36:57.909601 (+    11us) tablet.cc:1372] starting ApplyRowOperation cycle
0812 06:36:57.909789 (+   188us) tablet.cc:1383] finished ApplyRowOperation cycle
0812 06:36:57.909820 (+    31us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0812 06:36:57.909824 (+     4us) write_op.cc:312] APPLY: finished
0812 06:36:57.909894 (+    70us) log.cc:844] Serialized 77 byte log entry
0812 06:36:57.910050 (+   156us) write_op.cc:489] Releasing partition, row and schema locks
0812 06:36:57.910068 (+    18us) write_op.cc:454] Released schema lock
0812 06:36:57.910074 (+     6us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":1497,"num_ops":10,"prepare.queue_time_us":1109,"prepare.run_cpu_time_us":596,"prepare.run_wall_time_us":4977,"raft.queue_time_us":27,"raft.run_cpu_time_us":136,"raft.run_wall_time_us":136,"replication_time_us":2203786,"spinlock_wait_cycles":11648,"thread_start_us":84,"threads_started":1}]]}
I20260812 06:36:57.911103  9797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" candidate_term: 20 candidate_status { last_received { term: 19 index: 2790 } } ignore_live_leader: false dest_uuid: "2df4308c852240859ada53d216ab69ae" is_pre_election: true
I20260812 06:36:57.911301  6597 leader_election.cc:304] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 7 responses out of 7 voters: 3 yes votes; 4 no votes. yes voters: 222b20de5a1145c6b8a11df8374e8790, 42895a9bcba64aa484f6509605035d02, b037d17b21f8422fa5e41b70eb4dcd08; no voters: 2df4308c852240859ada53d216ab69ae, 7962f587ccd842ad8a3f6ed9a5d7707e, 8d90d61da62549208fe117b4384a469b, ec6c8d818e744cdbb6b0d15b321010e0
I20260812 06:36:57.911828  9315 leader_election.cc:304] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 7 responses out of 7 voters: 3 yes votes; 4 no votes. yes voters: 222b20de5a1145c6b8a11df8374e8790, 42895a9bcba64aa484f6509605035d02, b037d17b21f8422fa5e41b70eb4dcd08; no voters: 2df4308c852240859ada53d216ab69ae, 7962f587ccd842ad8a3f6ed9a5d7707e, 8d90d61da62549208fe117b4384a469b, ec6c8d818e744cdbb6b0d15b321010e0
I20260812 06:36:57.912338  6150 leader_election.cc:304] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 7 responses out of 7 voters: 3 yes votes; 4 no votes. yes voters: 8d90d61da62549208fe117b4384a469b, b037d17b21f8422fa5e41b70eb4dcd08, ec6c8d818e744cdbb6b0d15b321010e0; no voters: 222b20de5a1145c6b8a11df8374e8790, 2df4308c852240859ada53d216ab69ae, 42895a9bcba64aa484f6509605035d02, 7962f587ccd842ad8a3f6ed9a5d7707e
I20260812 06:36:57.912514  9421 raft_consensus.cc:1240] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [term 20 FOLLOWER]: Rejecting Update request from peer 2df4308c852240859ada53d216ab69ae for earlier term 19. Current term is 20. Ops: []
I20260812 06:36:57.912539  7320 raft_consensus.cc:1240] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08 [term 20 FOLLOWER]: Rejecting Update request from peer 2df4308c852240859ada53d216ab69ae for earlier term 19. Current term is 20. Ops: [19.2791-19.2887]
I20260812 06:36:57.912775 10320 raft_consensus.cc:2749] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260812 06:36:57.914067 10289 consensus_queue.cc:1059] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "222b20de5a1145c6b8a11df8374e8790" member_type: VOTER last_known_addr { host: "127.4.177.5" port: 43521 }, Status: INVALID_TERM, Last received: 19.2887, Next index: 2888, Last known committed idx: 2879, Time since last communication: 0.000s
I20260812 06:36:57.914244 10289 consensus_queue.cc:1059] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" member_type: VOTER last_known_addr { host: "127.4.177.7" port: 46299 }, Status: INVALID_TERM, Last received: 19.2790, Next index: 2791, Last known committed idx: 2789, Time since last communication: 0.000s
I20260812 06:36:57.914353 10289 raft_consensus.cc:3055] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [term 19 LEADER]: Stepping down as leader of term 19
I20260812 06:36:57.914395 10289 raft_consensus.cc:740] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 2df4308c852240859ada53d216ab69ae, State: Running, Role: LEADER
I20260812 06:36:57.914538 10334 raft_consensus.cc:2749] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260812 06:36:57.914597 10289 consensus_queue.cc:260] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2880, Committed index: 2880, Last appended: 19.2887, Last appended by leader: 2887, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df4308c852240859ada53d216ab69ae" member_type: VOTER last_known_addr { host: "127.4.177.6" port: 34269 } } peers { permanent_uuid: "42895a9bcba64aa484f6509605035d02" member_type: VOTER last_known_addr { host: "127.4.177.4" port: 38181 } } peers { permanent_uuid: "222b20de5a1145c6b8a11df8374e8790" member_type: VOTER last_known_addr { host: "127.4.177.5" port: 43521 } } peers { permanent_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 40897 } } peers { permanent_uuid: "8d90d61da62549208fe117b4384a469b" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 41539 } } peers { permanent_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" member_type: VOTER last_known_addr { host: "127.4.177.7" port: 46299 } } peers { permanent_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 41057 } }
I20260812 06:36:57.914963 10289 raft_consensus.cc:3060] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [term 19 FOLLOWER]: Advancing to term 20
I20260812 06:36:57.916126  6238 raft_consensus.cc:1240] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b [term 20 FOLLOWER]: Rejecting Update request from peer 2df4308c852240859ada53d216ab69ae for earlier term 19. Current term is 20. Ops: [19.2831-19.2887]
I20260812 06:36:57.916530  6048 raft_consensus.cc:1240] T 7065bb403e3047dd9a0853c49318ef1f P ec6c8d818e744cdbb6b0d15b321010e0 [term 20 FOLLOWER]: Rejecting Update request from peer 2df4308c852240859ada53d216ab69ae for earlier term 19. Current term is 20. Ops: [19.2757-19.2887]
W20260812 06:36:57.917164  9744 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:41528: Illegal state: replica 2df4308c852240859ada53d216ab69ae is not leader of this config: current role FOLLOWER
I20260812 06:36:57.917702  6703 raft_consensus.cc:1240] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [term 20 FOLLOWER]: Rejecting Update request from peer 2df4308c852240859ada53d216ab69ae for earlier term 19. Current term is 20. Ops: []
I20260812 06:36:57.918157  8805 raft_consensus.cc:1240] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [term 20 LEADER]: Rejecting Update request from peer 2df4308c852240859ada53d216ab69ae for earlier term 19. Current term is 20. Ops: [19.2881-19.2887]
I20260812 06:36:57.918818 10333 raft_consensus.cc:2749] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260812 06:36:57.922596  6623 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:42408: Illegal state: replica 42895a9bcba64aa484f6509605035d02 is not leader of this config: current role FOLLOWER
W20260812 06:36:57.934768  9359 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:46864: Illegal state: replica 222b20de5a1145c6b8a11df8374e8790 is not leader of this config: current role FOLLOWER
I20260812 06:36:57.942759  6238 raft_consensus.cc:1275] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b [term 20 FOLLOWER]: Refusing update from remote peer 7962f587ccd842ad8a3f6ed9a5d7707e: Log matching property violated. Preceding OpId in replica: term: 19 index: 2830. Preceding OpId from leader: term: 20 index: 2882. (index mismatch)
I20260812 06:36:57.942796  6048 raft_consensus.cc:1275] T 7065bb403e3047dd9a0853c49318ef1f P ec6c8d818e744cdbb6b0d15b321010e0 [term 20 FOLLOWER]: Refusing update from remote peer 7962f587ccd842ad8a3f6ed9a5d7707e: Log matching property violated. Preceding OpId in replica: term: 19 index: 2756. Preceding OpId from leader: term: 20 index: 2882. (index mismatch)
I20260812 06:36:57.943003  6703 raft_consensus.cc:1275] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [term 20 FOLLOWER]: Refusing update from remote peer 7962f587ccd842ad8a3f6ed9a5d7707e: Log matching property violated. Preceding OpId in replica: term: 19 index: 2887. Preceding OpId from leader: term: 20 index: 2882. (term mismatch)
I20260812 06:36:57.943084  6703 pending_rounds.cc:85] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02: Aborting all ops after (but not including) 2881
I20260812 06:36:57.943125  6703 pending_rounds.cc:107] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02: Aborting uncommitted WRITE_OP operation due to leader change: 19.2882
I20260812 06:36:57.943198  9421 raft_consensus.cc:1275] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [term 20 FOLLOWER]: Refusing update from remote peer 7962f587ccd842ad8a3f6ed9a5d7707e: Log matching property violated. Preceding OpId in replica: term: 19 index: 2887. Preceding OpId from leader: term: 20 index: 2882. (term mismatch)
I20260812 06:36:57.943198  9800 raft_consensus.cc:1275] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [term 20 FOLLOWER]: Refusing update from remote peer 7962f587ccd842ad8a3f6ed9a5d7707e: Log matching property violated. Preceding OpId in replica: term: 19 index: 2887. Preceding OpId from leader: term: 20 index: 2882. (term mismatch)
I20260812 06:36:57.943264  9800 pending_rounds.cc:85] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae: Aborting all ops after (but not including) 2881
I20260812 06:36:57.943271  9421 pending_rounds.cc:85] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790: Aborting all ops after (but not including) 2881
I20260812 06:36:57.943295  9800 pending_rounds.cc:107] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae: Aborting uncommitted WRITE_OP operation due to leader change: 19.2882
I20260812 06:36:57.943322  6703 pending_rounds.cc:107] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02: Aborting uncommitted WRITE_OP operation due to leader change: 19.2883
I20260812 06:36:57.943338  9421 pending_rounds.cc:107] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790: Aborting uncommitted WRITE_OP operation due to leader change: 19.2882
I20260812 06:36:57.943418  6703 pending_rounds.cc:107] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02: Aborting uncommitted WRITE_OP operation due to leader change: 19.2884
W20260812 06:36:57.943444  9800 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:41528: Aborted: Op aborted by new leader
I20260812 06:36:57.943500  6703 pending_rounds.cc:107] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02: Aborting uncommitted WRITE_OP operation due to leader change: 19.2885
I20260812 06:36:57.943564  9421 pending_rounds.cc:107] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790: Aborting uncommitted WRITE_OP operation due to leader change: 19.2883
I20260812 06:36:57.943588  6703 pending_rounds.cc:107] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02: Aborting uncommitted WRITE_OP operation due to leader change: 19.2886
I20260812 06:36:57.943585  9800 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:41528 (ReqId={client: 3067f3aef2a44cc4bb7f2ccd6acf83db, seq_no=2868, attempt_no=0}) took 2239 ms. Trace:
I20260812 06:36:57.943666  6703 pending_rounds.cc:107] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02: Aborting uncommitted WRITE_OP operation due to leader change: 19.2887
I20260812 06:36:57.943718  9421 pending_rounds.cc:107] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790: Aborting uncommitted WRITE_OP operation due to leader change: 19.2884
I20260812 06:36:57.943629  9800 rpcz_store.cc:276] 0812 06:36:55.704284 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 06:36:55.704323 (+    39us) service_pool.cc:224] Handling call
0812 06:36:57.943571 (+2239248us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0812 06:36:55.709365 (+     0us) write_op.cc:183] PREPARE: starting on tablet 7065bb403e3047dd9a0853c49318ef1f
0812 06:36:55.709433 (+    68us) write_op.cc:432] Acquiring schema lock in shared mode
0812 06:36:55.709436 (+     3us) write_op.cc:435] Acquired schema lock
0812 06:36:55.709438 (+     2us) tablet.cc:662] Decoding operations
0812 06:36:55.709542 (+   104us) write_op.cc:620] Acquiring the partition lock for write op
0812 06:36:55.709548 (+     6us) write_op.cc:641] Partition lock acquired for write op
0812 06:36:55.709551 (+     3us) tablet.cc:685] Acquiring locks for 10 operations
0812 06:36:55.709606 (+    55us) tablet.cc:701] Row locks acquired
0812 06:36:55.709607 (+     1us) write_op.cc:260] PREPARE: finished
0812 06:36:55.709630 (+    23us) write_op.cc:270] Start()
0812 06:36:55.709644 (+    14us) write_op.cc:276] Timestamp: P: 1786516615709570 usec, L: 0
0812 06:36:55.709645 (+     1us) op_driver.cc:348] REPLICATION: starting
0812 06:36:55.709725 (+    80us) log.cc:844] Serialized 696 byte log entry
0812 06:36:57.943389 (+2233664us) op_driver.cc:464] REPLICATION: finished
0812 06:36:57.943397 (+     8us) op_driver.cc:389] HandleFailure(Aborted: Op aborted by new leader)
0812 06:36:57.943401 (+     4us) write_op.cc:489] Releasing partition, row and schema locks
0812 06:36:57.943430 (+    29us) write_op.cc:454] Released schema lock
0812 06:36:57.943438 (+     8us) write_op.cc:335] FINISH: Op aborted
Metrics: {"child_traces":[["op",{"num_ops":10,"prepare.queue_time_us":4785,"prepare.run_cpu_time_us":427,"prepare.run_wall_time_us":426,"replication_time_us":2233678}]]}
I20260812 06:36:57.943929  9421 pending_rounds.cc:107] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790: Aborting uncommitted WRITE_OP operation due to leader change: 19.2885
I20260812 06:36:57.943938  9800 pending_rounds.cc:107] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae: Aborting uncommitted WRITE_OP operation due to leader change: 19.2883
W20260812 06:36:57.944048  9800 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:41528: Aborted: Op aborted by new leader
I20260812 06:36:57.944098  9421 pending_rounds.cc:107] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790: Aborting uncommitted WRITE_OP operation due to leader change: 19.2886
I20260812 06:36:57.944176  9800 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:41528 (ReqId={client: 3067f3aef2a44cc4bb7f2ccd6acf83db, seq_no=2867, attempt_no=0}) took 2239 ms. Trace:
I20260812 06:36:57.944247  9421 pending_rounds.cc:107] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790: Aborting uncommitted WRITE_OP operation due to leader change: 19.2887
I20260812 06:36:57.944218  9800 rpcz_store.cc:276] 0812 06:36:55.704215 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 06:36:55.704499 (+   284us) service_pool.cc:224] Handling call
0812 06:36:57.944167 (+2239668us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0812 06:36:55.709817 (+     0us) write_op.cc:183] PREPARE: starting on tablet 7065bb403e3047dd9a0853c49318ef1f
0812 06:36:55.709886 (+    69us) write_op.cc:432] Acquiring schema lock in shared mode
0812 06:36:55.709889 (+     3us) write_op.cc:435] Acquired schema lock
0812 06:36:55.709890 (+     1us) tablet.cc:662] Decoding operations
0812 06:36:55.709971 (+    81us) write_op.cc:620] Acquiring the partition lock for write op
0812 06:36:55.709976 (+     5us) write_op.cc:641] Partition lock acquired for write op
0812 06:36:55.709979 (+     3us) tablet.cc:685] Acquiring locks for 10 operations
0812 06:36:55.710040 (+    61us) tablet.cc:701] Row locks acquired
0812 06:36:55.710041 (+     1us) write_op.cc:260] PREPARE: finished
0812 06:36:55.710063 (+    22us) write_op.cc:270] Start()
0812 06:36:55.710077 (+    14us) write_op.cc:276] Timestamp: P: 1786516615710003 usec, L: 0
0812 06:36:55.710078 (+     1us) op_driver.cc:348] REPLICATION: starting
0812 06:36:55.710155 (+    77us) log.cc:844] Serialized 686 byte log entry
0812 06:36:57.944009 (+2233854us) op_driver.cc:464] REPLICATION: finished
0812 06:36:57.944014 (+     5us) op_driver.cc:389] HandleFailure(Aborted: Op aborted by new leader)
0812 06:36:57.944017 (+     3us) write_op.cc:489] Releasing partition, row and schema locks
0812 06:36:57.944040 (+    23us) write_op.cc:454] Released schema lock
0812 06:36:57.944045 (+     5us) write_op.cc:335] FINISH: Op aborted
Metrics: {"child_traces":[["op",{"num_ops":10,"prepare.queue_time_us":5069,"prepare.run_cpu_time_us":404,"prepare.run_wall_time_us":403,"replication_time_us":2233900}]]}
I20260812 06:36:57.944464  9800 pending_rounds.cc:107] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae: Aborting uncommitted WRITE_OP operation due to leader change: 19.2884
I20260812 06:36:57.944392 10346 consensus_queue.cc:1048] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [LEADER]: Connected to new peer: Peer: permanent_uuid: "42895a9bcba64aa484f6509605035d02" member_type: VOTER last_known_addr { host: "127.4.177.4" port: 38181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2881, Last known committed idx: 2879, Time since last communication: 0.000s
I20260812 06:36:57.944537 10346 consensus_queue.cc:1243] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [LEADER]: Peer 42895a9bcba64aa484f6509605035d02 log is divergent from this leader: its last log entry 19.2881 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2879
W20260812 06:36:57.944569  9800 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:41528: Aborted: Op aborted by new leader
I20260812 06:36:57.944617 10322 consensus_queue.cc:1048] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8d90d61da62549208fe117b4384a469b" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 41539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2881, Last known committed idx: 2830, Time since last communication: 0.000s
I20260812 06:36:57.944682  9800 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:41528 (ReqId={client: 3067f3aef2a44cc4bb7f2ccd6acf83db, seq_no=2869, attempt_no=0}) took 2240 ms. Trace:
I20260812 06:36:57.944727  9800 rpcz_store.cc:276] 0812 06:36:55.704668 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 06:36:55.704703 (+    35us) service_pool.cc:224] Handling call
0812 06:36:57.944674 (+2239971us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0812 06:36:55.710905 (+     0us) write_op.cc:183] PREPARE: starting on tablet 7065bb403e3047dd9a0853c49318ef1f
0812 06:36:55.710980 (+    75us) write_op.cc:432] Acquiring schema lock in shared mode
0812 06:36:55.710983 (+     3us) write_op.cc:435] Acquired schema lock
0812 06:36:55.710985 (+     2us) tablet.cc:662] Decoding operations
0812 06:36:55.711096 (+   111us) write_op.cc:620] Acquiring the partition lock for write op
0812 06:36:55.711102 (+     6us) write_op.cc:641] Partition lock acquired for write op
0812 06:36:55.711105 (+     3us) tablet.cc:685] Acquiring locks for 10 operations
0812 06:36:55.711168 (+    63us) tablet.cc:701] Row locks acquired
0812 06:36:55.711169 (+     1us) write_op.cc:260] PREPARE: finished
0812 06:36:55.711196 (+    27us) write_op.cc:270] Start()
0812 06:36:55.711213 (+    17us) write_op.cc:276] Timestamp: P: 1786516615711134 usec, L: 0
0812 06:36:55.711215 (+     2us) op_driver.cc:348] REPLICATION: starting
0812 06:36:55.711308 (+    93us) log.cc:844] Serialized 696 byte log entry
0812 06:36:57.944536 (+2233228us) op_driver.cc:464] REPLICATION: finished
0812 06:36:57.944541 (+     5us) op_driver.cc:389] HandleFailure(Aborted: Op aborted by new leader)
0812 06:36:57.944544 (+     3us) write_op.cc:489] Releasing partition, row and schema locks
0812 06:36:57.944561 (+    17us) write_op.cc:454] Released schema lock
0812 06:36:57.944566 (+     5us) write_op.cc:335] FINISH: Op aborted
Metrics: {"child_traces":[["op",{"num_ops":10,"prepare.queue_time_us":5372,"prepare.run_cpu_time_us":454,"prepare.run_wall_time_us":1070,"replication_time_us":2233292}]]}
I20260812 06:36:57.944989  9800 pending_rounds.cc:107] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae: Aborting uncommitted WRITE_OP operation due to leader change: 19.2885
W20260812 06:36:57.945112  9800 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:41528: Aborted: Op aborted by new leader
I20260812 06:36:57.945242  9800 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:41528 (ReqId={client: 3067f3aef2a44cc4bb7f2ccd6acf83db, seq_no=2870, attempt_no=0}) took 2240 ms. Trace:
I20260812 06:36:57.945287  9800 rpcz_store.cc:276] 0812 06:36:55.705118 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 06:36:55.705153 (+    35us) service_pool.cc:224] Handling call
0812 06:36:57.945230 (+2240077us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0812 06:36:55.711374 (+     0us) write_op.cc:183] PREPARE: starting on tablet 7065bb403e3047dd9a0853c49318ef1f
0812 06:36:55.711443 (+    69us) write_op.cc:432] Acquiring schema lock in shared mode
0812 06:36:55.711446 (+     3us) write_op.cc:435] Acquired schema lock
0812 06:36:55.711447 (+     1us) tablet.cc:662] Decoding operations
0812 06:36:55.711540 (+    93us) write_op.cc:620] Acquiring the partition lock for write op
0812 06:36:55.711545 (+     5us) write_op.cc:641] Partition lock acquired for write op
0812 06:36:55.711548 (+     3us) tablet.cc:685] Acquiring locks for 10 operations
0812 06:36:55.711601 (+    53us) tablet.cc:701] Row locks acquired
0812 06:36:55.711602 (+     1us) write_op.cc:260] PREPARE: finished
0812 06:36:55.711623 (+    21us) write_op.cc:270] Start()
0812 06:36:55.711637 (+    14us) write_op.cc:276] Timestamp: P: 1786516615711564 usec, L: 0
0812 06:36:55.711638 (+     1us) op_driver.cc:348] REPLICATION: starting
0812 06:36:55.711716 (+    78us) log.cc:844] Serialized 696 byte log entry
0812 06:36:57.945072 (+2233356us) op_driver.cc:464] REPLICATION: finished
0812 06:36:57.945078 (+     6us) op_driver.cc:389] HandleFailure(Aborted: Op aborted by new leader)
0812 06:36:57.945081 (+     3us) write_op.cc:489] Releasing partition, row and schema locks
0812 06:36:57.945102 (+    21us) write_op.cc:454] Released schema lock
0812 06:36:57.945108 (+     6us) write_op.cc:335] FINISH: Op aborted
Metrics: {"child_traces":[["op",{"num_ops":10,"prepare.queue_time_us":6021,"prepare.run_cpu_time_us":409,"prepare.run_wall_time_us":408,"replication_time_us":2233398}]]}
I20260812 06:36:57.945492  7320 raft_consensus.cc:1275] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08 [term 20 FOLLOWER]: Refusing update from remote peer 7962f587ccd842ad8a3f6ed9a5d7707e: Log matching property violated. Preceding OpId in replica: term: 19 index: 2790. Preceding OpId from leader: term: 20 index: 2882. (index mismatch)
I20260812 06:36:57.945559  9800 pending_rounds.cc:107] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae: Aborting uncommitted WRITE_OP operation due to leader change: 19.2886
W20260812 06:36:57.945683  9800 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:41528: Aborted: Op aborted by new leader
I20260812 06:36:57.945791  9800 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:41528 (ReqId={client: 3067f3aef2a44cc4bb7f2ccd6acf83db, seq_no=2871, attempt_no=0}) took 2240 ms. Trace:
I20260812 06:36:57.945875 10322 consensus_queue.cc:1048] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [LEADER]: Connected to new peer: Peer: permanent_uuid: "222b20de5a1145c6b8a11df8374e8790" member_type: VOTER last_known_addr { host: "127.4.177.5" port: 43521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2881, Last known committed idx: 2879, Time since last communication: 0.000s
I20260812 06:36:57.945832  9800 rpcz_store.cc:276] 0812 06:36:55.705433 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 06:36:55.705510 (+    77us) service_pool.cc:224] Handling call
0812 06:36:57.945780 (+2240270us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0812 06:36:55.711811 (+     0us) write_op.cc:183] PREPARE: starting on tablet 7065bb403e3047dd9a0853c49318ef1f
0812 06:36:55.711879 (+    68us) write_op.cc:432] Acquiring schema lock in shared mode
0812 06:36:55.711882 (+     3us) write_op.cc:435] Acquired schema lock
0812 06:36:55.711883 (+     1us) tablet.cc:662] Decoding operations
0812 06:36:55.711969 (+    86us) write_op.cc:620] Acquiring the partition lock for write op
0812 06:36:55.711982 (+    13us) write_op.cc:641] Partition lock acquired for write op
0812 06:36:55.711984 (+     2us) tablet.cc:685] Acquiring locks for 10 operations
0812 06:36:55.712028 (+    44us) tablet.cc:701] Row locks acquired
0812 06:36:55.712029 (+     1us) write_op.cc:260] PREPARE: finished
0812 06:36:55.712050 (+    21us) write_op.cc:270] Start()
0812 06:36:55.712064 (+    14us) write_op.cc:276] Timestamp: P: 1786516615711991 usec, L: 0
0812 06:36:55.712065 (+     1us) op_driver.cc:348] REPLICATION: starting
0812 06:36:55.712142 (+    77us) log.cc:844] Serialized 696 byte log entry
0812 06:36:57.945634 (+2233492us) op_driver.cc:464] REPLICATION: finished
0812 06:36:57.945640 (+     6us) op_driver.cc:389] HandleFailure(Aborted: Op aborted by new leader)
0812 06:36:57.945647 (+     7us) write_op.cc:489] Releasing partition, row and schema locks
0812 06:36:57.945672 (+    25us) write_op.cc:454] Released schema lock
0812 06:36:57.945679 (+     7us) write_op.cc:335] FINISH: Op aborted
Metrics: {"child_traces":[["op",{"num_ops":10,"prepare.queue_time_us":6033,"prepare.run_cpu_time_us":396,"prepare.run_wall_time_us":396,"replication_time_us":2233539}]]}
I20260812 06:36:57.945993 10322 consensus_queue.cc:1243] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [LEADER]: Peer 222b20de5a1145c6b8a11df8374e8790 log is divergent from this leader: its last log entry 19.2881 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2879
I20260812 06:36:57.946070  9800 pending_rounds.cc:107] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae: Aborting uncommitted WRITE_OP operation due to leader change: 19.2887
I20260812 06:36:57.946517 10343 consensus_queue.cc:1048] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [LEADER]: Connected to new peer: Peer: permanent_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" member_type: VOTER last_known_addr { host: "127.4.177.7" port: 46299 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2881, Last known committed idx: 2789, Time since last communication: 0.000s
W20260812 06:36:57.946632  9800 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:41528: Aborted: Op aborted by new leader
I20260812 06:36:57.946735  9800 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:41528 (ReqId={client: 3067f3aef2a44cc4bb7f2ccd6acf83db, seq_no=2872, attempt_no=0}) took 2229 ms. Trace:
I20260812 06:36:57.946770  9800 rpcz_store.cc:276] 0812 06:36:55.717129 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 06:36:55.717187 (+    58us) service_pool.cc:224] Handling call
0812 06:36:57.946727 (+2229540us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0812 06:36:55.717556 (+     0us) write_op.cc:183] PREPARE: starting on tablet 7065bb403e3047dd9a0853c49318ef1f
0812 06:36:55.717623 (+    67us) write_op.cc:432] Acquiring schema lock in shared mode
0812 06:36:55.717626 (+     3us) write_op.cc:435] Acquired schema lock
0812 06:36:55.717627 (+     1us) tablet.cc:662] Decoding operations
0812 06:36:55.717695 (+    68us) write_op.cc:620] Acquiring the partition lock for write op
0812 06:36:55.717699 (+     4us) write_op.cc:641] Partition lock acquired for write op
0812 06:36:55.717701 (+     2us) tablet.cc:685] Acquiring locks for 10 operations
0812 06:36:55.717751 (+    50us) tablet.cc:701] Row locks acquired
0812 06:36:55.717752 (+     1us) write_op.cc:260] PREPARE: finished
0812 06:36:55.717776 (+    24us) write_op.cc:270] Start()
0812 06:36:55.717792 (+    16us) write_op.cc:276] Timestamp: P: 1786516615717715 usec, L: 0
0812 06:36:55.717793 (+     1us) op_driver.cc:348] REPLICATION: starting
0812 06:36:55.717884 (+    91us) log.cc:844] Serialized 696 byte log entry
0812 06:36:57.946591 (+2228707us) op_driver.cc:464] REPLICATION: finished
0812 06:36:57.946597 (+     6us) op_driver.cc:389] HandleFailure(Aborted: Op aborted by new leader)
0812 06:36:57.946600 (+     3us) write_op.cc:489] Releasing partition, row and schema locks
0812 06:36:57.946623 (+    23us) write_op.cc:454] Released schema lock
0812 06:36:57.946629 (+     6us) write_op.cc:335] FINISH: Op aborted
Metrics: {"child_traces":[["op",{"num_ops":10,"prepare.queue_time_us":56,"prepare.run_cpu_time_us":446,"prepare.run_wall_time_us":445,"raft.queue_time_us":128,"raft.run_cpu_time_us":336,"raft.run_wall_time_us":336,"replication_time_us":2228761,"spinlock_wait_cycles":896}]]}
I20260812 06:36:57.947237  6703 raft_consensus.cc:1217] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [term 20 FOLLOWER]: Deduplicated request from leader. Original: 19.2879->[19.2880-20.2882]   Dedup: 19.2880->[20.2881-20.2882]
I20260812 06:36:57.947315  6703 pending_rounds.cc:85] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02: Aborting all ops after (but not including) 2880
I20260812 06:36:57.947347  6703 pending_rounds.cc:107] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02: Aborting uncommitted WRITE_OP operation due to leader change: 19.2881
I20260812 06:36:57.951642 10322 consensus_queue.cc:1048] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [LEADER]: Connected to new peer: Peer: permanent_uuid: "2df4308c852240859ada53d216ab69ae" member_type: VOTER last_known_addr { host: "127.4.177.6" port: 34269 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2881, Last known committed idx: 2880, Time since last communication: 0.000s
I20260812 06:36:57.951751 10322 consensus_queue.cc:1243] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [LEADER]: Peer 2df4308c852240859ada53d216ab69ae log is divergent from this leader: its last log entry 19.2881 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2880
I20260812 06:36:57.951985 10322 consensus_queue.cc:1048] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 41057 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2881, Last known committed idx: 2754, Time since last communication: 0.000s
I20260812 06:36:57.955421  9421 raft_consensus.cc:1217] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [term 20 FOLLOWER]: Deduplicated request from leader. Original: 19.2879->[19.2880-20.2882]   Dedup: 19.2880->[20.2881-20.2882]
I20260812 06:36:57.955545  9421 pending_rounds.cc:85] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790: Aborting all ops after (but not including) 2880
I20260812 06:36:57.955571  9421 pending_rounds.cc:107] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790: Aborting uncommitted WRITE_OP operation due to leader change: 19.2881
I20260812 06:36:57.962199  9800 pending_rounds.cc:85] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae: Aborting all ops after (but not including) 2880
I20260812 06:36:57.962393  9800 pending_rounds.cc:107] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae: Aborting uncommitted WRITE_OP operation due to leader change: 19.2881
W20260812 06:36:57.962595  9800 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:41528: Aborted: Op aborted by new leader
I20260812 06:36:57.962800  9800 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:41528 (ReqId={client: 3067f3aef2a44cc4bb7f2ccd6acf83db, seq_no=2865, attempt_no=0}) took 2260 ms. Trace:
I20260812 06:36:57.962909  9800 rpcz_store.cc:276] 0812 06:36:55.702179 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 06:36:55.702971 (+   792us) service_pool.cc:224] Handling call
0812 06:36:57.962786 (+2259815us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0812 06:36:55.708783 (+     0us) write_op.cc:183] PREPARE: starting on tablet 7065bb403e3047dd9a0853c49318ef1f
0812 06:36:55.708861 (+    78us) write_op.cc:432] Acquiring schema lock in shared mode
0812 06:36:55.708864 (+     3us) write_op.cc:435] Acquired schema lock
0812 06:36:55.708866 (+     2us) tablet.cc:662] Decoding operations
0812 06:36:55.708969 (+   103us) write_op.cc:620] Acquiring the partition lock for write op
0812 06:36:55.708976 (+     7us) write_op.cc:641] Partition lock acquired for write op
0812 06:36:55.708982 (+     6us) tablet.cc:685] Acquiring locks for 10 operations
0812 06:36:55.709056 (+    74us) tablet.cc:701] Row locks acquired
0812 06:36:55.709057 (+     1us) write_op.cc:260] PREPARE: finished
0812 06:36:55.709088 (+    31us) write_op.cc:270] Start()
0812 06:36:55.709112 (+    24us) write_op.cc:276] Timestamp: P: 1786516615709024 usec, L: 0
0812 06:36:55.709114 (+     2us) op_driver.cc:348] REPLICATION: starting
0812 06:36:55.709254 (+   140us) log.cc:844] Serialized 696 byte log entry
0812 06:36:57.962544 (+2253290us) op_driver.cc:464] REPLICATION: finished
0812 06:36:57.962552 (+     8us) op_driver.cc:389] HandleFailure(Aborted: Op aborted by new leader)
0812 06:36:57.962556 (+     4us) write_op.cc:489] Releasing partition, row and schema locks
0812 06:36:57.962581 (+    25us) write_op.cc:454] Released schema lock
0812 06:36:57.962590 (+     9us) write_op.cc:335] FINISH: Op aborted
Metrics: {"child_traces":[["op",{"num_ops":10,"prepare.queue_time_us":5570,"prepare.run_cpu_time_us":559,"prepare.run_wall_time_us":559,"raft.queue_time_us":46,"raft.run_cpu_time_us":139,"raft.run_wall_time_us":139,"replication_time_us":2253380,"spinlock_wait_cycles":90496}]]}
I20260812 06:36:58.699083  4804 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskCnZu03/build/debug/bin/kudu with pid 8639
W20260812 06:36:58.717245  7401 meta_cache.cc:302] tablet 7065bb403e3047dd9a0853c49318ef1f: replica 7962f587ccd842ad8a3f6ed9a5d7707e (127.4.177.3:40897) has failed: Network error: recv got EOF from 127.4.177.3:40897 (error 108)
W20260812 06:36:58.726735  9309 connection.cc:466] server connection from 127.4.177.3:54637 torn down before Call kudu.consensus.ConsensusService.UpdateConsensus from 127.4.177.3:54637 (request call id 53) could send its response
W20260812 06:36:58.731464  7210 connection.cc:466] server connection from 127.4.177.3:47507 torn down before Call kudu.consensus.ConsensusService.UpdateConsensus from 127.4.177.3:47507 (request call id 18) could send its response
W20260812 06:36:58.733222  6048 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.4.177.3:42963 (request call id 7) took 779 ms (client timeout 1000 ms). Trace:
W20260812 06:36:58.733309  6048 rpcz_store.cc:269] 0812 06:36:57.953754 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 06:36:57.953805 (+    51us) service_pool.cc:224] Handling call
0812 06:36:57.955621 (+  1816us) raft_consensus.cc:1471] Updating replica for 126 ops
0812 06:36:57.956608 (+   987us) raft_consensus.cc:1534] Early marking committed up to index 2756
0812 06:36:57.958474 (+  1866us) raft_consensus.cc:1539] Triggering prepare for 126 ops
0812 06:36:57.958581 (+   107us) write_op.cc:270] Start()
0812 06:36:57.958602 (+    21us) write_op.cc:276] Timestamp: P: 1786516615371495 usec, L: 0
0812 06:36:57.958832 (+   230us) write_op.cc:270] Start()
0812 06:36:57.958850 (+    18us) write_op.cc:276] Timestamp: P: 1786516615375928 usec, L: 0
0812 06:36:57.958946 (+    96us) write_op.cc:270] Start()
0812 06:36:57.958958 (+    12us) write_op.cc:276] Timestamp: P: 1786516615376428 usec, L: 0
0812 06:36:57.959036 (+    78us) write_op.cc:270] Start()
0812 06:36:57.959048 (+    12us) write_op.cc:276] Timestamp: P: 1786516615376923 usec, L: 0
0812 06:36:57.959134 (+    86us) write_op.cc:270] Start()
0812 06:36:57.959146 (+    12us) write_op.cc:276] Timestamp: P: 1786516615377370 usec, L: 0
0812 06:36:57.959225 (+    79us) write_op.cc:270] Start()
0812 06:36:57.959237 (+    12us) write_op.cc:276] Timestamp: P: 1786516615378410 usec, L: 0
0812 06:36:57.959315 (+    78us) write_op.cc:270] Start()
0812 06:36:57.959360 (+    45us) write_op.cc:276] Timestamp: P: 1786516615394106 usec, L: 0
0812 06:36:57.959447 (+    87us) write_op.cc:270] Start()
0812 06:36:57.959461 (+    14us) write_op.cc:276] Timestamp: P: 1786516615395334 usec, L: 0
0812 06:36:57.959548 (+    87us) write_op.cc:270] Start()
0812 06:36:57.959560 (+    12us) write_op.cc:276] Timestamp: P: 1786516615395954 usec, L: 0
0812 06:36:57.959637 (+    77us) write_op.cc:270] Start()
0812 06:36:57.959649 (+    12us) write_op.cc:276] Timestamp: P: 1786516615396612 usec, L: 0
0812 06:36:57.959725 (+    76us) write_op.cc:270] Start()
0812 06:36:57.959737 (+    12us) write_op.cc:276] Timestamp: P: 1786516615397758 usec, L: 0
0812 06:36:57.959820 (+    83us) write_op.cc:270] Start()
0812 06:36:57.959832 (+    12us) write_op.cc:276] Timestamp: P: 1786516615398391 usec, L: 0
0812 06:36:57.959919 (+    87us) write_op.cc:270] Start()
0812 06:36:57.959932 (+    13us) write_op.cc:276] Timestamp: P: 1786516615398854 usec, L: 0
0812 06:36:57.960015 (+    83us) write_op.cc:270] Start()
0812 06:36:57.960027 (+    12us) write_op.cc:276] Timestamp: P: 1786516615399297 usec, L: 0
0812 06:36:57.960108 (+    81us) write_op.cc:270] Start()
0812 06:36:57.960120 (+    12us) write_op.cc:276] Timestamp: P: 1786516615413509 usec, L: 0
0812 06:36:57.960223 (+   103us) write_op.cc:270] Start()
0812 06:36:57.960235 (+    12us) write_op.cc:276] Timestamp: P: 1786516615418607 usec, L: 0
0812 06:36:57.960320 (+    85us) write_op.cc:270] Start()
0812 06:36:57.960331 (+    11us) write_op.cc:276] Timestamp: P: 1786516615426735 usec, L: 0
0812 06:36:57.960421 (+    90us) write_op.cc:270] Start()
0812 06:36:57.960433 (+    12us) write_op.cc:276] Timestamp: P: 1786516615430106 usec, L: 0
0812 06:36:57.960512 (+    79us) write_op.cc:270] Start()
0812 06:36:57.960525 (+    13us) write_op.cc:276] Timestamp: P: 1786516615433757 usec, L: 0
0812 06:36:57.960602 (+    77us) write_op.cc:270] Start()
0812 06:36:57.960614 (+    12us) write_op.cc:276] Timestamp: P: 1786516615434307 usec, L: 0
0812 06:36:57.960694 (+    80us) write_op.cc:270] Start()
0812 06:36:57.960706 (+    12us) write_op.cc:276] Timestamp: P: 1786516615435757 usec, L: 0
0812 06:36:57.960783 (+    77us) write_op.cc:270] Start()
0812 06:36:57.960795 (+    12us) write_op.cc:276] Timestamp: P: 1786516615436321 usec, L: 0
0812 06:36:57.960874 (+    79us) write_op.cc:270] Start()
0812 06:36:57.960886 (+    12us) write_op.cc:276] Timestamp: P: 1786516615442846 usec, L: 0
0812 06:36:57.960970 (+    84us) write_op.cc:270] Start()
0812 06:36:57.960982 (+    12us) write_op.cc:276] Timestamp: P: 1786516615443354 usec, L: 0
0812 06:36:57.961075 (+    93us) write_op.cc:270] Start()
0812 06:36:57.961087 (+    12us) write_op.cc:276] Timestamp: P: 1786516615443827 usec, L: 0
0812 06:36:57.961167 (+    80us) write_op.cc:270] Start()
0812 06:36:57.961179 (+    12us) write_op.cc:276] Timestamp: P: 1786516615450578 usec, L: 0
0812 06:36:57.961263 (+    84us) write_op.cc:270] Start()
0812 06:36:57.961278 (+    15us) write_op.cc:276] Timestamp: P: 1786516615458047 usec, L: 0
0812 06:36:57.961373 (+    95us) write_op.cc:270] Start()
0812 06:36:57.961387 (+    14us) write_op.cc:276] Timestamp: P: 1786516615461481 usec, L: 0
0812 06:36:57.961473 (+    86us) write_op.cc:270] Start()
0812 06:36:57.961485 (+    12us) write_op.cc:276] Timestamp: P: 1786516615462485 usec, L: 0
0812 06:36:57.961570 (+    85us) write_op.cc:270] Start()
0812 06:36:57.961582 (+    12us) write_op.cc:276] Timestamp: P: 1786516615462957 usec, L: 0
0812 06:36:57.961658 (+    76us) write_op.cc:270] Start()
0812 06:36:57.961673 (+    15us) write_op.cc:276] Timestamp: P: 1786516615463675 usec, L: 0
0812 06:36:57.961754 (+    81us) write_op.cc:270] Start()
0812 06:36:57.961766 (+    12us) write_op.cc:276] Timestamp: P: 1786516615464614 usec, L: 0
0812 06:36:57.965665 (+  3899us) write_op.cc:270] Start()
0812 06:36:57.965694 (+    29us) write_op.cc:276] Timestamp: P: 1786516615465192 usec, L: 0
0812 06:36:57.966109 (+   415us) write_op.cc:270] Start()
0812 06:36:57.966128 (+    19us) write_op.cc:276] Timestamp: P: 1786516615474315 usec, L: 0
0812 06:36:57.966275 (+   147us) write_op.cc:270] Start()
0812 06:36:57.966293 (+    18us) write_op.cc:276] Timestamp: P: 1786516615487250 usec, L: 0
0812 06:36:57.966755 (+   462us) write_op.cc:270] Start()
0812 06:36:57.966779 (+    24us) write_op.cc:276] Timestamp: P: 1786516615487712 usec, L: 0
0812 06:36:57.966898 (+   119us) write_op.cc:270] Start()
0812 06:36:57.966913 (+    15us) write_op.cc:276] Timestamp: P: 1786516615488156 usec, L: 0
0812 06:36:57.967009 (+    96us) write_op.cc:270] Start()
0812 06:36:57.967027 (+    18us) write_op.cc:276] Timestamp: P: 1786516615488620 usec, L: 0
0812 06:36:57.967128 (+   101us) write_op.cc:270] Start()
0812 06:36:57.967147 (+    19us) write_op.cc:276] Timestamp: P: 1786516615489090 usec, L: 0
0812 06:36:57.967246 (+    99us) write_op.cc:270] Start()
0812 06:36:57.967262 (+    16us) write_op.cc:276] Timestamp: P: 1786516615489554 usec, L: 0
0812 06:36:57.967352 (+    90us) write_op.cc:270] Start()
0812 06:36:57.967365 (+    13us) write_op.cc:276] Timestamp: P: 1786516615491413 usec, L: 0
0812 06:36:57.967469 (+   104us) write_op.cc:270] Start()
0812 06:36:57.967484 (+    15us) write_op.cc:276] Timestamp: P: 1786516615496156 usec, L: 0
0812 06:36:57.967582 (+    98us) write_op.cc:270] Start()
0812 06:36:57.967601 (+    19us) write_op.cc:276] Timestamp: P: 1786516615507693 usec, L: 0
0812 06:36:57.967677 (+    76us) write_op.cc:270] Start()
0812 06:36:57.967690 (+    13us) write_op.cc:276] Timestamp: P: 1786516615508909 usec, L: 0
0812 06:36:57.967771 (+    81us) write_op.cc:270] Start()
0812 06:36:57.967790 (+    19us) write_op.cc:276] Timestamp: P: 1786516615509729 usec, L: 0
0812 06:36:57.967878 (+    88us) write_op.cc:270] Start()
0812 06:36:57.967890 (+    12us) write_op.cc:276] Timestamp: P: 1786516615510704 usec, L: 0
0812 06:36:57.967981 (+    91us) write_op.cc:270] Start()
0812 06:36:57.968002 (+    21us) write_op.cc:276] Timestamp: P: 1786516615511525 usec, L: 0
0812 06:36:57.968117 (+   115us) write_op.cc:270] Start()
0812 06:36:57.968132 (+    15us) write_op.cc:276] Timestamp: P: 1786516615513172 usec, L: 0
0812 06:36:57.968227 (+    95us) write_op.cc:270] Start()
0812 06:36:57.968241 (+    14us) write_op.cc:276] Timestamp: P: 1786516615513652 usec, L: 0
0812 06:36:57.968330 (+    89us) write_op.cc:270] Start()
0812 06:36:57.968347 (+    17us) write_op.cc:276] Timestamp: P: 1786516615514106 usec, L: 0
0812 06:36:57.968451 (+   104us) write_op.cc:270] Start()
0812 06:36:57.968468 (+    17us) write_op.cc:276] Timestamp: P: 1786516615529316 usec, L: 0
0812 06:36:57.968555 (+    87us) write_op.cc:270] Start()
0812 06:36:57.968567 (+    12us) write_op.cc:276] Timestamp: P: 1786516615530047 usec, L: 0
0812 06:36:57.968652 (+    85us) write_op.cc:270] Start()
0812 06:36:57.968665 (+    13us) write_op.cc:276] Timestamp: P: 1786516615530752 usec, L: 0
0812 06:36:57.968758 (+    93us) write_op.cc:270] Start()
0812 06:36:57.968772 (+    14us) write_op.cc:276] Timestamp: P: 1786516615531203 usec, L: 0
0812 06:36:57.968875 (+   103us) write_op.cc:270] Start()
0812 06:36:57.968893 (+    18us) write_op.cc:276] Timestamp: P: 1786516615531644 usec, L: 0
0812 06:36:57.968999 (+   106us) write_op.cc:270] Start()
0812 06:36:57.969013 (+    14us) write_op.cc:276] Timestamp: P: 1786516615532152 usec, L: 0
0812 06:36:57.969099 (+    86us) write_op.cc:270] Start()
0812 06:36:57.969112 (+    13us) write_op.cc:276] Timestamp: P: 1786516615532620 usec, L: 0
0812 06:36:57.969203 (+    91us) write_op.cc:270] Start()
0812 06:36:57.969223 (+    20us) write_op.cc:276] Timestamp: P: 1786516615536080 usec, L: 0
0812 06:36:57.969308 (+    85us) write_op.cc:270] Start()
0812 06:36:57.969325 (+    17us) write_op.cc:276] Timestamp: P: 1786516615550116 usec, L: 0
0812 06:36:57.969429 (+   104us) write_op.cc:270] Start()
0812 06:36:57.969454 (+    25us) write_op.cc:276] Timestamp: P: 1786516615552728 usec, L: 0
0812 06:36:57.969538 (+    84us) write_op.cc:270] Start()
0812 06:36:57.969554 (+    16us) write_op.cc:276] Timestamp: P: 1786516615554699 usec, L: 0
0812 06:36:57.969660 (+   106us) write_op.cc:270] Start()
0812 06:36:57.969678 (+    18us) write_op.cc:276] Timestamp: P: 1786516615555213 usec, L: 0
0812 06:36:57.969780 (+   102us) write_op.cc:270] Start()
0812 06:36:57.969798 (+    18us) write_op.cc:276] Timestamp: P: 1786516615555704 usec, L: 0
0812 06:36:57.969892 (+    94us) write_op.cc:270] Start()
0812 06:36:57.969907 (+    15us) write_op.cc:276] Timestamp: P: 1786516615558573 usec, L: 0
0812 06:36:57.970003 (+    96us) write_op.cc:270] Start()
0812 06:36:57.970019 (+    16us) write_op.cc:276] Timestamp: P: 1786516615559022 usec, L: 0
0812 06:36:57.970114 (+    95us) write_op.cc:270] Start()
0812 06:36:57.970130 (+    16us) write_op.cc:276] Timestamp: P: 1786516615559523 usec, L: 0
0812 06:36:57.970237 (+   107us) write_op.cc:270] Start()
0812 06:36:57.970254 (+    17us) write_op.cc:276] Timestamp: P: 1786516615574162 usec, L: 0
0812 06:36:57.970351 (+    97us) write_op.cc:270] Start()
0812 06:36:57.970368 (+    17us) write_op.cc:276] Timestamp: P: 1786516615574726 usec, L: 0
0812 06:36:57.970478 (+   110us) write_op.cc:270] Start()
0812 06:36:57.970495 (+    17us) write_op.cc:276] Timestamp: P: 1786516615575235 usec, L: 0
0812 06:36:57.970594 (+    99us) write_op.cc:270] Start()
0812 06:36:57.970607 (+    13us) write_op.cc:276] Timestamp: P: 1786516615577581 usec, L: 0
0812 06:36:57.970692 (+    85us) write_op.cc:270] Start()
0812 06:36:57.970708 (+    16us) write_op.cc:276] Timestamp: P: 1786516615578110 usec, L: 0
0812 06:36:57.970801 (+    93us) write_op.cc:270] Start()
0812 06:36:57.970813 (+    12us) write_op.cc:276] Timestamp: P: 1786516615578639 usec, L: 0
0812 06:36:57.970891 (+    78us) write_op.cc:270] Start()
0812 06:36:57.970904 (+    13us) write_op.cc:276] Timestamp: P: 1786516615579148 usec, L: 0
0812 06:36:57.970999 (+    95us) write_op.cc:270] Start()
0812 06:36:57.971013 (+    14us) write_op.cc:276] Timestamp: P: 1786516615579662 usec, L: 0
0812 06:36:57.971102 (+    89us) write_op.cc:270] Start()
0812 06:36:57.971120 (+    18us) write_op.cc:276] Timestamp: P: 1786516615593409 usec, L: 0
0812 06:36:57.971201 (+    81us) write_op.cc:270] Start()
0812 06:36:57.971213 (+    12us) write_op.cc:276] Timestamp: P: 1786516615593942 usec, L: 0
0812 06:36:57.971291 (+    78us) write_op.cc:270] Start()
0812 06:36:57.971303 (+    12us) write_op.cc:276] Timestamp: P: 1786516615594433 usec, L: 0
0812 06:36:57.974730 (+  3427us) write_op.cc:270] Start()
0812 06:36:57.974748 (+    18us) write_op.cc:276] Timestamp: P: 1786516615594881 usec, L: 0
0812 06:36:57.974852 (+   104us) write_op.cc:270] Start()
0812 06:36:57.974866 (+    14us) write_op.cc:276] Timestamp: P: 1786516615595430 usec, L: 0
0812 06:36:57.974974 (+   108us) write_op.cc:270] Start()
0812 06:36:57.974995 (+    21us) write_op.cc:276] Timestamp: P: 1786516615597713 usec, L: 0
0812 06:36:57.975088 (+    93us) write_op.cc:270] Start()
0812 06:36:57.975103 (+    15us) write_op.cc:276] Timestamp: P: 1786516615598120 usec, L: 0
0812 06:36:57.975192 (+    89us) write_op.cc:270] Start()
0812 06:36:57.975211 (+    19us) write_op.cc:276] Timestamp: P: 1786516615598633 usec, L: 0
0812 06:36:57.975308 (+    97us) write_op.cc:270] Start()
0812 06:36:57.975325 (+    17us) write_op.cc:276] Timestamp: P: 1786516615610246 usec, L: 0
0812 06:36:57.975417 (+    92us) write_op.cc:270] Start()
0812 06:36:57.975429 (+    12us) write_op.cc:276] Timestamp: P: 1786516615611012 usec, L: 0
0812 06:36:57.975515 (+    86us) write_op.cc:270] Start()
0812 06:36:57.975528 (+    13us) write_op.cc:276] Timestamp: P: 1786516615611641 usec, L: 0
0812 06:36:57.975611 (+    83us) write_op.cc:270] Start()
0812 06:36:57.975624 (+    13us) write_op.cc:276] Timestamp: P: 1786516615612970 usec, L: 0
0812 06:36:57.975705 (+    81us) write_op.cc:270] Start()
0812 06:36:57.975719 (+    14us) write_op.cc:276] Timestamp: P: 1786516615613412 usec, L: 0
0812 06:36:57.975808 (+    89us) write_op.cc:270] Start()
0812 06:36:57.975821 (+    13us) write_op.cc:276] Timestamp: P: 1786516615613863 usec, L: 0
0812 06:36:57.975904 (+    83us) write_op.cc:270] Start()
0812 06:36:57.975916 (+    12us) write_op.cc:276] Timestamp: P: 1786516615614335 usec, L: 0
0812 06:36:57.976004 (+    88us) write_op.cc:270] Start()
0812 06:36:57.976017 (+    13us) write_op.cc:276] Timestamp: P: 1786516615614784 usec, L: 0
0812 06:36:57.976111 (+    94us) write_op.cc:270] Start()
0812 06:36:57.978941 (+  2830us) write_op.cc:276] Timestamp: P: 1786516615624579 usec, L: 0
0812 06:36:57.979057 (+   116us) write_op.cc:270] Start()
0812 06:36:57.979075 (+    18us) write_op.cc:276] Timestamp: P: 1786516615625488 usec, L: 0
0812 06:36:57.979172 (+    97us) write_op.cc:270] Start()
0812 06:36:57.979191 (+    19us) write_op.cc:276] Timestamp: P: 1786516615626360 usec, L: 0
0812 06:36:57.979293 (+   102us) write_op.cc:270] Start()
0812 06:36:57.979311 (+    18us) write_op.cc:276] Timestamp: P: 1786516615627058 usec, L: 0
0812 06:36:57.979462 (+   151us) write_op.cc:270] Start()
0812 06:36:57.979483 (+    21us) write_op.cc:276] Timestamp: P: 1786516615627727 usec, L: 0
0812 06:36:57.979597 (+   114us) write_op.cc:270] Start()
0812 06:36:57.979614 (+    17us) write_op.cc:276] Timestamp: P: 1786516615628427 usec, L: 0
0812 06:36:57.979709 (+    95us) write_op.cc:270] Start()
0812 06:36:57.979723 (+    14us) write_op.cc:276] Timestamp: P: 1786516615629112 usec, L: 0
0812 06:36:57.979812 (+    89us) write_op.cc:270] Start()
0812 06:36:57.979825 (+    13us) write_op.cc:276] Timestamp: P: 1786516615633717 usec, L: 0
0812 06:36:57.979918 (+    93us) write_op.cc:270] Start()
0812 06:36:57.979933 (+    15us) write_op.cc:276] Timestamp: P: 1786516615644124 usec, L: 0
0812 06:36:57.980023 (+    90us) write_op.cc:270] Start()
0812 06:36:57.980038 (+    15us) write_op.cc:276] Timestamp: P: 1786516615644652 usec, L: 0
0812 06:36:57.980135 (+    97us) write_op.cc:270] Start()
0812 06:36:57.980148 (+    13us) write_op.cc:276] Timestamp: P: 1786516615645169 usec, L: 0
0812 06:36:57.980240 (+    92us) write_op.cc:270] Start()
0812 06:36:57.980256 (+    16us) write_op.cc:276] Timestamp: P: 1786516615645664 usec, L: 0
0812 06:36:57.980351 (+    95us) write_op.cc:270] Start()
0812 06:36:57.980364 (+    13us) write_op.cc:276] Timestamp: P: 1786516615646144 usec, L: 0
0812 06:36:57.980458 (+    94us) write_op.cc:270] Start()
0812 06:36:57.980471 (+    13us) write_op.cc:276] Timestamp: P: 1786516615646657 usec, L: 0
0812 06:36:57.980550 (+    79us) write_op.cc:270] Start()
0812 06:36:57.980562 (+    12us) write_op.cc:276] Timestamp: P: 1786516615647160 usec, L: 0
0812 06:36:57.980639 (+    77us) write_op.cc:270] Start()
0812 06:36:57.980652 (+    13us) write_op.cc:276] Timestamp: P: 1786516615652493 usec, L: 0
0812 06:36:57.980732 (+    80us) write_op.cc:270] Start()
0812 06:36:57.980744 (+    12us) write_op.cc:276] Timestamp: P: 1786516615667155 usec, L: 0
0812 06:36:57.980822 (+    78us) write_op.cc:270] Start()
0812 06:36:57.980834 (+    12us) write_op.cc:276] Timestamp: P: 1786516615667768 usec, L: 0
0812 06:36:57.980913 (+    79us) write_op.cc:270] Start()
0812 06:36:57.980925 (+    12us) write_op.cc:276] Timestamp: P: 1786516615668280 usec, L: 0
0812 06:36:57.981010 (+    85us) write_op.cc:270] Start()
0812 06:36:57.981022 (+    12us) write_op.cc:276] Timestamp: P: 1786516615668782 usec, L: 0
0812 06:36:57.981106 (+    84us) write_op.cc:270] Start()
0812 06:36:57.981119 (+    13us) write_op.cc:276] Timestamp: P: 1786516615669364 usec, L: 0
0812 06:36:57.981206 (+    87us) write_op.cc:270] Start()
0812 06:36:57.981218 (+    12us) write_op.cc:276] Timestamp: P: 1786516615669843 usec, L: 0
0812 06:36:57.981301 (+    83us) write_op.cc:270] Start()
0812 06:36:57.981313 (+    12us) write_op.cc:276] Timestamp: P: 1786516615670941 usec, L: 0
0812 06:36:57.981392 (+    79us) write_op.cc:270] Start()
0812 06:36:57.981415 (+    23us) write_op.cc:276] Timestamp: P: 1786516615673085 usec, L: 0
0812 06:36:57.981505 (+    90us) write_op.cc:270] Start()
0812 06:36:57.981518 (+    13us) write_op.cc:276] Timestamp: P: 1786516615676071 usec, L: 0
0812 06:36:57.981602 (+    84us) write_op.cc:270] Start()
0812 06:36:57.981615 (+    13us) write_op.cc:276] Timestamp: P: 1786516615684031 usec, L: 0
0812 06:36:57.981696 (+    81us) write_op.cc:270] Start()
0812 06:36:57.981708 (+    12us) write_op.cc:276] Timestamp: P: 1786516615684654 usec, L: 0
0812 06:36:57.981840 (+   132us) write_op.cc:270] Start()
0812 06:36:57.981855 (+    15us) write_op.cc:276] Timestamp: P: 1786516615685091 usec, L: 0
0812 06:36:57.981945 (+    90us) write_op.cc:270] Start()
0812 06:36:57.981958 (+    13us) write_op.cc:276] Timestamp: P: 1786516615686225 usec, L: 0
0812 06:36:57.982039 (+    81us) write_op.cc:270] Start()
0812 06:36:57.982051 (+    12us) write_op.cc:276] Timestamp: P: 1786516615686697 usec, L: 0
0812 06:36:57.982136 (+    85us) write_op.cc:270] Start()
0812 06:36:57.982153 (+    17us) write_op.cc:276] Timestamp: P: 1786516615688086 usec, L: 0
0812 06:36:57.982318 (+   165us) write_op.cc:270] Start()
0812 06:36:57.982335 (+    17us) write_op.cc:276] Timestamp: P: 1786516615689456 usec, L: 0
0812 06:36:57.982455 (+   120us) write_op.cc:270] Start()
0812 06:36:57.982471 (+    16us) write_op.cc:276] Timestamp: P: 1786516615696604 usec, L: 0
0812 06:36:57.982568 (+    97us) write_op.cc:270] Start()
0812 06:36:57.982587 (+    19us) write_op.cc:276] Timestamp: P: 1786516615704067 usec, L: 0
0812 06:36:57.982851 (+   264us) write_op.cc:270] Start()
0812 06:36:57.982877 (+    26us) write_op.cc:276] Timestamp: P: 1786516617941237 usec, L: 0
0812 06:36:57.985740 (+  2863us) log.cc:844] Serialized 86707 byte log entry
0812 06:36:57.985822 (+    82us) raft_consensus.cc:1645] Marking committed up to 2878
0812 06:36:57.989747 (+  3925us) raft_consensus.cc:1697] Filling consensus response to leader.
0812 06:36:57.992079 (+  2332us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0812 06:36:58.733139 (+741060us) raft_consensus.cc:1685] finished
0812 06:36:58.733141 (+     2us) raft_consensus.cc:1691] UpdateReplicas() finished
0812 06:36:58.733205 (+    64us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0812 06:36:57.981789 (+     0us) write_op.cc:183] PREPARE: starting on tablet 7065bb403e3047dd9a0853c49318ef1f
0812 06:36:57.981879 (+    90us) write_op.cc:432] Acquiring schema lock in shared mode
0812 06:36:57.981883 (+     4us) write_op.cc:435] Acquired schema lock
0812 06:36:57.981884 (+     1us) tablet.cc:662] Decoding operations
0812 06:36:57.981980 (+    96us) write_op.cc:620] Acquiring the partition lock for write op
0812 06:36:57.981989 (+     9us) write_op.cc:641] Partition lock acquired for write op
0812 06:36:57.981992 (+     3us) tablet.cc:685] Acquiring locks for 10 operations
0812 06:36:57.982057 (+    65us) tablet.cc:701] Row locks acquired
0812 06:36:57.982059 (+     2us) write_op.cc:260] PREPARE: finished
0812 06:36:57.985863 (+  3804us) op_driver.cc:464] REPLICATION: finished
0812 06:36:57.985990 (+   127us) write_op.cc:301] APPLY: starting
0812 06:36:57.986003 (+    13us) tablet.cc:1367] starting BulkCheckPresence
0812 06:36:57.986032 (+    29us) tablet.cc:1370] finished BulkCheckPresence
0812 06:36:57.986034 (+     2us) tablet.cc:1372] starting ApplyRowOperation cycle
0812 06:36:57.986201 (+   167us) tablet.cc:1383] finished ApplyRowOperation cycle
0812 06:36:57.986648 (+   447us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0812 06:36:57.986658 (+    10us) write_op.cc:312] APPLY: finished
0812 06:36:57.986719 (+    61us) log.cc:844] Serialized 77 byte log entry
0812 06:36:57.986768 (+    49us) write_op.cc:489] Releasing partition, row and schema locks
0812 06:36:58.719448 (+732680us) spinlock_profiling.cc:243] Waited 732 ms on lock 0x558235a00f98. stack: 00007f90242e6d35 00007f90242e7325 00007f9023df382d 0000558232269b18 0000558232269b6b 000055823226d9f2 00007f902695c033 00007f902695af13 00007f9026900a6e 00007f90268ff785 00007f90268fde49 00007f90268eb16c 00007f90268eab95 00007f90268e99ca 00007f90268ec95b 00005582322f944d
0812 06:36:58.719460 (+    12us) write_op.cc:454] Released schema lock
0812 06:36:58.719473 (+    13us) write_op.cc:341] FINISH: Updating metrics
0812 06:36:58.719598 (+   125us) write_op.cc:489] Releasing partition, row and schema locks
0812 06:36:58.719603 (+     5us) write_op.cc:454] Released schema lock
Related trace 'op':
0812 06:36:57.982128 (+     0us) write_op.cc:183] PREPARE: starting on tablet 7065bb403e3047dd9a0853c49318ef1f
0812 06:36:57.982192 (+    64us) write_op.cc:432] Acquiring schema lock in shared mode
0812 06:36:57.982196 (+     4us) write_op.cc:435] Acquired schema lock
0812 06:36:57.982197 (+     1us) tablet.cc:662] Decoding operations
0812 06:36:57.982303 (+   106us) write_op.cc:620] Acquiring the partition lock for write op
0812 06:36:57.982311 (+     8us) write_op.cc:641] Partition lock acquired for write op
0812 06:36:57.982315 (+     4us) tablet.cc:685] Acquiring locks for 10 operations
0812 06:36:57.982374 (+    59us) tablet.cc:701] Row locks acquired
0812 06:36:57.982376 (+     2us) write_op.cc:260] PREPARE: finished
0812 06:36:57.985917 (+  3541us) op_driver.cc:464] REPLICATION: finished
0812 06:36:57.986909 (+   992us) write_op.cc:301] APPLY: starting
0812 06:36:57.986919 (+    10us) tablet.cc:1367] starting BulkCheckPresence
0812 06:36:57.986947 (+    28us) tablet.cc:1370] finished BulkCheckPresence
0812 06:36:57.986948 (+     1us) tablet.cc:1372] starting ApplyRowOperation cycle
0812 06:36:57.987116 (+   168us) tablet.cc:1383] finished ApplyRowOperation cycle
0812 06:36:57.987455 (+   339us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0812 06:36:57.987469 (+    14us) write_op.cc:312] APPLY: finished
0812 06:36:57.987533 (+    64us) log.cc:844] Serialized 77 byte log entry
0812 06:36:57.987583 (+    50us) write_op.cc:489] Releasing partition, row and schema locks
0812 06:36:58.722384 (+734801us) spinlock_profiling.cc:243] Waited 735 ms on lock 0x558235a00f98. stack: 00007f90242e6d35 00007f90242e7325 00007f9023df382d 0000558232269b18 0000558232269b6b 000055823226d9f2 00007f902695c033 00007f902695af13 00007f9026900a6e 00007f90268ff785 00007f90268fde49 00007f90268eb16c 00007f90268eab95 00007f90268e99ca 00007f90268ec95b 00005582322f944d
0812 06:36:58.722389 (+     5us) write_op.cc:454] Released schema lock
0812 06:36:58.722394 (+     5us) write_op.cc:341] FINISH: Updating metrics
0812 06:36:58.722463 (+    69us) write_op.cc:489] Releasing partition, row and schema locks
0812 06:36:58.722465 (+     2us) write_op.cc:454] Released schema lock
Related trace 'op':
0812 06:36:57.982438 (+     0us) write_op.cc:183] PREPARE: starting on tablet 7065bb403e3047dd9a0853c49318ef1f
0812 06:36:57.982510 (+    72us) write_op.cc:432] Acquiring schema lock in shared mode
0812 06:36:57.982513 (+     3us) write_op.cc:435] Acquired schema lock
0812 06:36:57.982515 (+     2us) tablet.cc:662] Decoding operations
0812 06:36:57.982591 (+    76us) write_op.cc:620] Acquiring the partition lock for write op
0812 06:36:57.982598 (+     7us) write_op.cc:641] Partition lock acquired for write op
0812 06:36:57.982600 (+     2us) tablet.cc:685] Acquiring locks for 10 operations
0812 06:36:57.982654 (+    54us) tablet.cc:701] Row locks acquired
0812 06:36:57.982655 (+     1us) write_op.cc:260] PREPARE: finished
0812 06:36:57.987728 (+  5073us) op_driver.cc:464] REPLICATION: finished
0812 06:36:57.999232 (+ 11504us) write_op.cc:301] APPLY: starting
0812 06:36:57.999250 (+    18us) tablet.cc:1367] starting BulkCheckPresence
0812 06:36:57.999285 (+    35us) tablet.cc:1370] finished BulkCheckPresence
0812 06:36:57.999286 (+     1us) tablet.cc:1372] starting ApplyRowOperation cycle
0812 06:36:57.999535 (+   249us) tablet.cc:1383] finished ApplyRowOperation cycle
0812 06:36:57.999592 (+    57us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0812 06:36:57.999598 (+     6us) write_op.cc:312] APPLY: finished
0812 06:36:57.999670 (+    72us) log.cc:844] Serialized 77 byte log entry
0812 06:36:58.004639 (+  4969us) write_op.cc:489] Releasing partition, row and schema locks
0812 06:36:58.726594 (+721955us) spinlock_profiling.cc:243] Waited 722 ms on lock 0x558235a00f98. stack: 00007f90242e6d35 00007f90242e7325 00007f9023df382d 0000558232269b18 0000558232269b6b 000055823226d9f2 00007f902695c033 00007f902695af13 00007f9026900a6e 00007f90268ff785 00007f90268fde49 00007f90268eb16c 00007f90268eab95 00007f90268e99ca 00007f90268ec95b 00005582322f944d
0812 06:36:58.726600 (+     6us) write_op.cc:454] Released schema lock
0812 06:36:58.726607 (+     7us) write_op.cc:341] FINISH: Updating metrics
0812 06:36:58.726680 (+    73us) write_op.cc:489] Releasing partition, row and schema locks
0812 06:36:58.726684 (+     4us) write_op.cc:454] Released schema lock
Related trace 'op':
0812 06:36:57.982721 (+     0us) write_op.cc:183] PREPARE: starting on tablet 7065bb403e3047dd9a0853c49318ef1f
0812 06:36:57.982780 (+    59us) write_op.cc:432] Acquiring schema lock in shared mode
0812 06:36:57.982782 (+     2us) write_op.cc:435] Acquired schema lock
0812 06:36:57.982783 (+     1us) tablet.cc:662] Decoding operations
0812 06:36:57.982862 (+    79us) write_op.cc:620] Acquiring the partition lock for write op
0812 06:36:57.982869 (+     7us) write_op.cc:641] Partition lock acquired for write op
0812 06:36:57.982872 (+     3us) tablet.cc:685] Acquiring locks for 10 operations
0812 06:36:57.982931 (+    59us) tablet.cc:701] Row locks acquired
0812 06:36:57.982932 (+     1us) write_op.cc:260] PREPARE: finished
0812 06:36:57.987891 (+  4959us) op_driver.cc:464] REPLICATION: finished
0812 06:36:58.002778 (+ 14887us) write_op.cc:301] APPLY: starting
0812 06:36:58.002793 (+    15us) tablet.cc:1367] starting BulkCheckPresence
0812 06:36:58.002826 (+    33us) tablet.cc:1370] finished BulkCheckPresence
0812 06:36:58.002828 (+     2us) tablet.cc:1372] starting ApplyRowOperation cycle
0812 06:36:58.004313 (+  1485us) tablet.cc:1383] finished ApplyRowOperation cycle
0812 06:36:58.004370 (+    57us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0812 06:36:58.004375 (+     5us) write_op.cc:312] APPLY: finished
0812 06:36:58.004455 (+    80us) log.cc:844] Serialized 77 byte log entry
0812 06:36:58.004492 (+    37us) write_op.cc:489] Releasing partition, row and schema locks
0812 06:36:58.726342 (+721850us) spinlock_profiling.cc:243] Waited 722 ms on lock 0x558235a00f98. stack: 00007f90242e6d35 00007f90242e7325 00007f9023df382d 0000558232269b18 0000558232269b6b 000055823226d9f2 00007f902695c033 00007f902695af13 00007f9026900a6e 00007f90268ff785 00007f90268fde49 00007f90268eb16c 00007f90268eab95 00007f90268e99ca 00007f90268ec95b 00005582322f944d
0812 06:36:58.726348 (+     6us) write_op.cc:454] Released schema lock
0812 06:36:58.726353 (+     5us) write_op.cc:341] FINISH: Updating metrics
0812 06:36:58.726437 (+    84us) write_op.cc:489] Releasing partition, row and schema locks
0812 06:36:58.726440 (+     3us) write_op.cc:454] Released schema lock
Related trace 'op':
0812 06:36:57.983020 (+     0us) write_op.cc:183] PREPARE: starting on tablet 7065bb403e3047dd9a0853c49318ef1f
0812 06:36:57.983082 (+    62us) write_op.cc:432] Acquiring schema lock in shared mode
0812 06:36:57.983086 (+     4us) write_op.cc:435] Acquired schema lock
0812 06:36:57.983087 (+     1us) tablet.cc:662] Decoding operations
0812 06:36:57.983160 (+    73us) write_op.cc:620] Acquiring the partition lock for write op
0812 06:36:57.983167 (+     7us) write_op.cc:641] Partition lock acquired for write op
0812 06:36:57.983170 (+     3us) tablet.cc:685] Acquiring locks for 10 operations
0812 06:36:57.983369 (+   199us) tablet.cc:701] Row locks acquired
0812 06:36:57.983371 (+     2us) write_op.cc:260] PREPARE: finished
0812 06:36:57.988036 (+  4665us) op_driver.cc:464] REPLICATION: finished
0812 06:36:58.719683 (+731647us) write_op.cc:301] APPLY: starting
0812 06:36:58.719702 (+    19us) tablet.cc:1367] starting BulkCheckPresence
0812 06:36:58.719756 (+    54us) tablet.cc:1370] finished BulkCheckPresence
0812 06:36:58.719759 (+     3us) tablet.cc:1372] starting ApplyRowOperation cycle
0812 06
W20260812 06:36:58.767059  5939 connection.cc:466] server connection from 127.4.177.3:42963 torn down before Call kudu.consensus.ConsensusService.UpdateConsensus from 127.4.177.3:42963 (request call id 7) could send its response
I20260812 06:36:59.750849 10427 raft_consensus.cc:493] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 7962f587ccd842ad8a3f6ed9a5d7707e)
I20260812 06:36:59.751012 10427 raft_consensus.cc:515] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df4308c852240859ada53d216ab69ae" member_type: VOTER last_known_addr { host: "127.4.177.6" port: 34269 } } peers { permanent_uuid: "42895a9bcba64aa484f6509605035d02" member_type: VOTER last_known_addr { host: "127.4.177.4" port: 38181 } } peers { permanent_uuid: "222b20de5a1145c6b8a11df8374e8790" member_type: VOTER last_known_addr { host: "127.4.177.5" port: 43521 } } peers { permanent_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 40897 } } peers { permanent_uuid: "8d90d61da62549208fe117b4384a469b" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 41539 } } peers { permanent_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" member_type: VOTER last_known_addr { host: "127.4.177.7" port: 46299 } } peers { permanent_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 41057 } }
I20260812 06:36:59.751853 10427 leader_election.cc:290] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 2df4308c852240859ada53d216ab69ae (127.4.177.6:34269), 42895a9bcba64aa484f6509605035d02 (127.4.177.4:38181), 222b20de5a1145c6b8a11df8374e8790 (127.4.177.5:43521), 7962f587ccd842ad8a3f6ed9a5d7707e (127.4.177.3:40897), b037d17b21f8422fa5e41b70eb4dcd08 (127.4.177.7:46299), ec6c8d818e744cdbb6b0d15b321010e0 (127.4.177.1:41057)
I20260812 06:36:59.752336  6686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "8d90d61da62549208fe117b4384a469b" candidate_term: 21 candidate_status { last_received { term: 20 index: 2971 } } ignore_live_leader: false dest_uuid: "42895a9bcba64aa484f6509605035d02" is_pre_election: true
I20260812 06:36:59.752334  7320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "8d90d61da62549208fe117b4384a469b" candidate_term: 21 candidate_status { last_received { term: 20 index: 2971 } } ignore_live_leader: false dest_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" is_pre_election: true
I20260812 06:36:59.752323  9797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "8d90d61da62549208fe117b4384a469b" candidate_term: 21 candidate_status { last_received { term: 20 index: 2971 } } ignore_live_leader: false dest_uuid: "2df4308c852240859ada53d216ab69ae" is_pre_election: true
I20260812 06:36:59.752856  6048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "8d90d61da62549208fe117b4384a469b" candidate_term: 21 candidate_status { last_received { term: 20 index: 2971 } } ignore_live_leader: false dest_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" is_pre_election: true
I20260812 06:36:59.753018  9421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "8d90d61da62549208fe117b4384a469b" candidate_term: 21 candidate_status { last_received { term: 20 index: 2971 } } ignore_live_leader: false dest_uuid: "222b20de5a1145c6b8a11df8374e8790" is_pre_election: true
W20260812 06:36:59.753213  6150 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.4.177.3:40897: connect: Connection refused (error 111) [suppressed 1 similar messages]
I20260812 06:36:59.753885  6150 leader_election.cc:304] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 5 responses out of 7 voters: 1 yes votes; 4 no votes. yes voters: 8d90d61da62549208fe117b4384a469b; no voters: 222b20de5a1145c6b8a11df8374e8790, 2df4308c852240859ada53d216ab69ae, 42895a9bcba64aa484f6509605035d02, b037d17b21f8422fa5e41b70eb4dcd08
I20260812 06:36:59.754074 10427 raft_consensus.cc:2749] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260812 06:36:59.754135  6150 leader_election.cc:336] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 7962f587ccd842ad8a3f6ed9a5d7707e (127.4.177.3:40897): Network error: Client connection negotiation failed: client connection to 127.4.177.3:40897: connect: Connection refused (error 111)
I20260812 06:36:59.880064 10432 raft_consensus.cc:493] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 7962f587ccd842ad8a3f6ed9a5d7707e)
I20260812 06:36:59.880226 10432 raft_consensus.cc:515] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df4308c852240859ada53d216ab69ae" member_type: VOTER last_known_addr { host: "127.4.177.6" port: 34269 } } peers { permanent_uuid: "42895a9bcba64aa484f6509605035d02" member_type: VOTER last_known_addr { host: "127.4.177.4" port: 38181 } } peers { permanent_uuid: "222b20de5a1145c6b8a11df8374e8790" member_type: VOTER last_known_addr { host: "127.4.177.5" port: 43521 } } peers { permanent_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 40897 } } peers { permanent_uuid: "8d90d61da62549208fe117b4384a469b" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 41539 } } peers { permanent_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" member_type: VOTER last_known_addr { host: "127.4.177.7" port: 46299 } } peers { permanent_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 41057 } }
I20260812 06:36:59.880764 10433 raft_consensus.cc:493] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 7962f587ccd842ad8a3f6ed9a5d7707e)
I20260812 06:36:59.880879 10433 raft_consensus.cc:515] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df4308c852240859ada53d216ab69ae" member_type: VOTER last_known_addr { host: "127.4.177.6" port: 34269 } } peers { permanent_uuid: "42895a9bcba64aa484f6509605035d02" member_type: VOTER last_known_addr { host: "127.4.177.4" port: 38181 } } peers { permanent_uuid: "222b20de5a1145c6b8a11df8374e8790" member_type: VOTER last_known_addr { host: "127.4.177.5" port: 43521 } } peers { permanent_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 40897 } } peers { permanent_uuid: "8d90d61da62549208fe117b4384a469b" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 41539 } } peers { permanent_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" member_type: VOTER last_known_addr { host: "127.4.177.7" port: 46299 } } peers { permanent_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 41057 } }
I20260812 06:36:59.881273 10432 leader_election.cc:290] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 2df4308c852240859ada53d216ab69ae (127.4.177.6:34269), 222b20de5a1145c6b8a11df8374e8790 (127.4.177.5:43521), 7962f587ccd842ad8a3f6ed9a5d7707e (127.4.177.3:40897), 8d90d61da62549208fe117b4384a469b (127.4.177.2:41539), b037d17b21f8422fa5e41b70eb4dcd08 (127.4.177.7:46299), ec6c8d818e744cdbb6b0d15b321010e0 (127.4.177.1:41057)
I20260812 06:36:59.881521  9797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "42895a9bcba64aa484f6509605035d02" candidate_term: 21 candidate_status { last_received { term: 20 index: 3028 } } ignore_live_leader: false dest_uuid: "2df4308c852240859ada53d216ab69ae" is_pre_election: true
I20260812 06:36:59.881669  9797 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 42895a9bcba64aa484f6509605035d02 in term 20.
I20260812 06:36:59.881686  7320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "42895a9bcba64aa484f6509605035d02" candidate_term: 21 candidate_status { last_received { term: 20 index: 3028 } } ignore_live_leader: false dest_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" is_pre_election: true
I20260812 06:36:59.881870  9421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "42895a9bcba64aa484f6509605035d02" candidate_term: 21 candidate_status { last_received { term: 20 index: 3028 } } ignore_live_leader: false dest_uuid: "222b20de5a1145c6b8a11df8374e8790" is_pre_election: true
W20260812 06:36:59.882426  6597 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.4.177.3:40897: connect: Connection refused (error 111) [suppressed 2 similar messages]
I20260812 06:36:59.882418  6241 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "42895a9bcba64aa484f6509605035d02" candidate_term: 21 candidate_status { last_received { term: 20 index: 3028 } } ignore_live_leader: false dest_uuid: "8d90d61da62549208fe117b4384a469b" is_pre_election: true
I20260812 06:36:59.882566  6241 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 42895a9bcba64aa484f6509605035d02 in term 20.
I20260812 06:36:59.882680  6686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "2df4308c852240859ada53d216ab69ae" candidate_term: 21 candidate_status { last_received { term: 20 index: 3028 } } ignore_live_leader: false dest_uuid: "42895a9bcba64aa484f6509605035d02" is_pre_election: true
I20260812 06:36:59.882805  6686 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2df4308c852240859ada53d216ab69ae in term 20.
I20260812 06:36:59.882781  6048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "42895a9bcba64aa484f6509605035d02" candidate_term: 21 candidate_status { last_received { term: 20 index: 3028 } } ignore_live_leader: false dest_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" is_pre_election: true
W20260812 06:36:59.883240  6597 leader_election.cc:336] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 7962f587ccd842ad8a3f6ed9a5d7707e (127.4.177.3:40897): Network error: Client connection negotiation failed: client connection to 127.4.177.3:40897: connect: Connection refused (error 111)
I20260812 06:36:59.883315  6597 leader_election.cc:304] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 7 responses out of 7 voters: 3 yes votes; 4 no votes. yes voters: 2df4308c852240859ada53d216ab69ae, 42895a9bcba64aa484f6509605035d02, 8d90d61da62549208fe117b4384a469b; no voters: 222b20de5a1145c6b8a11df8374e8790, 7962f587ccd842ad8a3f6ed9a5d7707e, b037d17b21f8422fa5e41b70eb4dcd08, ec6c8d818e744cdbb6b0d15b321010e0
I20260812 06:36:59.883380  9421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "2df4308c852240859ada53d216ab69ae" candidate_term: 21 candidate_status { last_received { term: 20 index: 3028 } } ignore_live_leader: false dest_uuid: "222b20de5a1145c6b8a11df8374e8790" is_pre_election: true
I20260812 06:36:59.883677 10432 raft_consensus.cc:2749] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260812 06:36:59.883730  7320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "2df4308c852240859ada53d216ab69ae" candidate_term: 21 candidate_status { last_received { term: 20 index: 3028 } } ignore_live_leader: false dest_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" is_pre_election: true
I20260812 06:36:59.883901  6241 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "2df4308c852240859ada53d216ab69ae" candidate_term: 21 candidate_status { last_received { term: 20 index: 3028 } } ignore_live_leader: false dest_uuid: "8d90d61da62549208fe117b4384a469b" is_pre_election: true
I20260812 06:36:59.884022  6241 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2df4308c852240859ada53d216ab69ae in term 20.
I20260812 06:36:59.884039 10433 leader_election.cc:290] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 42895a9bcba64aa484f6509605035d02 (127.4.177.4:38181), 222b20de5a1145c6b8a11df8374e8790 (127.4.177.5:43521), 7962f587ccd842ad8a3f6ed9a5d7707e (127.4.177.3:40897), 8d90d61da62549208fe117b4384a469b (127.4.177.2:41539), b037d17b21f8422fa5e41b70eb4dcd08 (127.4.177.7:46299), ec6c8d818e744cdbb6b0d15b321010e0 (127.4.177.1:41057)
W20260812 06:36:59.884227  9714 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.4.177.3:40897: connect: Connection refused (error 111)
I20260812 06:36:59.884405  6048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "2df4308c852240859ada53d216ab69ae" candidate_term: 21 candidate_status { last_received { term: 20 index: 3028 } } ignore_live_leader: false dest_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" is_pre_election: true
W20260812 06:36:59.885145  9714 leader_election.cc:336] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 7962f587ccd842ad8a3f6ed9a5d7707e (127.4.177.3:40897): Network error: Client connection negotiation failed: client connection to 127.4.177.3:40897: connect: Connection refused (error 111)
I20260812 06:36:59.885212  9714 leader_election.cc:304] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 7 responses out of 7 voters: 3 yes votes; 4 no votes. yes voters: 2df4308c852240859ada53d216ab69ae, 42895a9bcba64aa484f6509605035d02, 8d90d61da62549208fe117b4384a469b; no voters: 222b20de5a1145c6b8a11df8374e8790, 7962f587ccd842ad8a3f6ed9a5d7707e, b037d17b21f8422fa5e41b70eb4dcd08, ec6c8d818e744cdbb6b0d15b321010e0
I20260812 06:36:59.885346 10433 raft_consensus.cc:2749] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260812 06:37:00.218808 10442 raft_consensus.cc:493] T 7065bb403e3047dd9a0853c49318ef1f P ec6c8d818e744cdbb6b0d15b321010e0 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 7962f587ccd842ad8a3f6ed9a5d7707e)
I20260812 06:37:00.218963 10442 raft_consensus.cc:515] T 7065bb403e3047dd9a0853c49318ef1f P ec6c8d818e744cdbb6b0d15b321010e0 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df4308c852240859ada53d216ab69ae" member_type: VOTER last_known_addr { host: "127.4.177.6" port: 34269 } } peers { permanent_uuid: "42895a9bcba64aa484f6509605035d02" member_type: VOTER last_known_addr { host: "127.4.177.4" port: 38181 } } peers { permanent_uuid: "222b20de5a1145c6b8a11df8374e8790" member_type: VOTER last_known_addr { host: "127.4.177.5" port: 43521 } } peers { permanent_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 40897 } } peers { permanent_uuid: "8d90d61da62549208fe117b4384a469b" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 41539 } } peers { permanent_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" member_type: VOTER last_known_addr { host: "127.4.177.7" port: 46299 } } peers { permanent_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 41057 } }
I20260812 06:37:00.219725 10442 leader_election.cc:290] T 7065bb403e3047dd9a0853c49318ef1f P ec6c8d818e744cdbb6b0d15b321010e0 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 2df4308c852240859ada53d216ab69ae (127.4.177.6:34269), 42895a9bcba64aa484f6509605035d02 (127.4.177.4:38181), 222b20de5a1145c6b8a11df8374e8790 (127.4.177.5:43521), 7962f587ccd842ad8a3f6ed9a5d7707e (127.4.177.3:40897), 8d90d61da62549208fe117b4384a469b (127.4.177.2:41539), b037d17b21f8422fa5e41b70eb4dcd08 (127.4.177.7:46299)
I20260812 06:37:00.220208  7320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" candidate_term: 21 candidate_status { last_received { term: 20 index: 2882 } } ignore_live_leader: false dest_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" is_pre_election: true
I20260812 06:37:00.220235  6686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" candidate_term: 21 candidate_status { last_received { term: 20 index: 2882 } } ignore_live_leader: false dest_uuid: "42895a9bcba64aa484f6509605035d02" is_pre_election: true
I20260812 06:37:00.220235  9797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" candidate_term: 21 candidate_status { last_received { term: 20 index: 2882 } } ignore_live_leader: false dest_uuid: "2df4308c852240859ada53d216ab69ae" is_pre_election: true
I20260812 06:37:00.220407  6686 raft_consensus.cc:2410] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ec6c8d818e744cdbb6b0d15b321010e0 for term 21 because replica has last-logged OpId of term: 20 index: 3028, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 2882.
I20260812 06:37:00.220414  9797 raft_consensus.cc:2410] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ec6c8d818e744cdbb6b0d15b321010e0 for term 21 because replica has last-logged OpId of term: 20 index: 3028, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 2882.
I20260812 06:37:00.220806  9421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" candidate_term: 21 candidate_status { last_received { term: 20 index: 2882 } } ignore_live_leader: false dest_uuid: "222b20de5a1145c6b8a11df8374e8790" is_pre_election: true
W20260812 06:37:00.220968  5945 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.4.177.3:40897: connect: Connection refused (error 111)
I20260812 06:37:00.221078  6241 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" candidate_term: 21 candidate_status { last_received { term: 20 index: 2882 } } ignore_live_leader: false dest_uuid: "8d90d61da62549208fe117b4384a469b" is_pre_election: true
I20260812 06:37:00.221266  5945 leader_election.cc:304] T 7065bb403e3047dd9a0853c49318ef1f P ec6c8d818e744cdbb6b0d15b321010e0 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 5 responses out of 7 voters: 1 yes votes; 4 no votes. yes voters: ec6c8d818e744cdbb6b0d15b321010e0; no voters: 222b20de5a1145c6b8a11df8374e8790, 2df4308c852240859ada53d216ab69ae, 42895a9bcba64aa484f6509605035d02, b037d17b21f8422fa5e41b70eb4dcd08
I20260812 06:37:00.221276  6241 raft_consensus.cc:2410] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ec6c8d818e744cdbb6b0d15b321010e0 for term 21 because replica has last-logged OpId of term: 20 index: 2971, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 2882.
I20260812 06:37:00.221454 10442 raft_consensus.cc:2749] T 7065bb403e3047dd9a0853c49318ef1f P ec6c8d818e744cdbb6b0d15b321010e0 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260812 06:37:00.222028  5945 leader_election.cc:336] T 7065bb403e3047dd9a0853c49318ef1f P ec6c8d818e744cdbb6b0d15b321010e0 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 7962f587ccd842ad8a3f6ed9a5d7707e (127.4.177.3:40897): Network error: Client connection negotiation failed: client connection to 127.4.177.3:40897: connect: Connection refused (error 111)
I20260812 06:37:00.226859 10447 raft_consensus.cc:493] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 7962f587ccd842ad8a3f6ed9a5d7707e)
I20260812 06:37:00.226994 10447 raft_consensus.cc:515] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df4308c852240859ada53d216ab69ae" member_type: VOTER last_known_addr { host: "127.4.177.6" port: 34269 } } peers { permanent_uuid: "42895a9bcba64aa484f6509605035d02" member_type: VOTER last_known_addr { host: "127.4.177.4" port: 38181 } } peers { permanent_uuid: "222b20de5a1145c6b8a11df8374e8790" member_type: VOTER last_known_addr { host: "127.4.177.5" port: 43521 } } peers { permanent_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 40897 } } peers { permanent_uuid: "8d90d61da62549208fe117b4384a469b" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 41539 } } peers { permanent_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" member_type: VOTER last_known_addr { host: "127.4.177.7" port: 46299 } } peers { permanent_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 41057 } }
I20260812 06:37:00.227675 10447 leader_election.cc:290] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 2df4308c852240859ada53d216ab69ae (127.4.177.6:34269), 42895a9bcba64aa484f6509605035d02 (127.4.177.4:38181), 7962f587ccd842ad8a3f6ed9a5d7707e (127.4.177.3:40897), 8d90d61da62549208fe117b4384a469b (127.4.177.2:41539), b037d17b21f8422fa5e41b70eb4dcd08 (127.4.177.7:46299), ec6c8d818e744cdbb6b0d15b321010e0 (127.4.177.1:41057)
I20260812 06:37:00.227907  9797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "222b20de5a1145c6b8a11df8374e8790" candidate_term: 21 candidate_status { last_received { term: 20 index: 3020 } } ignore_live_leader: false dest_uuid: "2df4308c852240859ada53d216ab69ae" is_pre_election: true
I20260812 06:37:00.228010  7320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "222b20de5a1145c6b8a11df8374e8790" candidate_term: 21 candidate_status { last_received { term: 20 index: 3020 } } ignore_live_leader: false dest_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" is_pre_election: true
I20260812 06:37:00.228091  9797 raft_consensus.cc:2410] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 222b20de5a1145c6b8a11df8374e8790 for term 21 because replica has last-logged OpId of term: 20 index: 3028, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 3020.
I20260812 06:37:00.228374  6686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "222b20de5a1145c6b8a11df8374e8790" candidate_term: 21 candidate_status { last_received { term: 20 index: 3020 } } ignore_live_leader: false dest_uuid: "42895a9bcba64aa484f6509605035d02" is_pre_election: true
I20260812 06:37:00.228554  6686 raft_consensus.cc:2410] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 222b20de5a1145c6b8a11df8374e8790 for term 21 because replica has last-logged OpId of term: 20 index: 3028, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 3020.
W20260812 06:37:00.228538  9315 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.4.177.3:40897: connect: Connection refused (error 111)
I20260812 06:37:00.228562  6241 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "222b20de5a1145c6b8a11df8374e8790" candidate_term: 21 candidate_status { last_received { term: 20 index: 3020 } } ignore_live_leader: false dest_uuid: "8d90d61da62549208fe117b4384a469b" is_pre_election: true
I20260812 06:37:00.228694  6241 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 222b20de5a1145c6b8a11df8374e8790 in term 20.
I20260812 06:37:00.228711  6048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "222b20de5a1145c6b8a11df8374e8790" candidate_term: 21 candidate_status { last_received { term: 20 index: 3020 } } ignore_live_leader: false dest_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" is_pre_election: true
I20260812 06:37:00.228826  6048 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P ec6c8d818e744cdbb6b0d15b321010e0 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 222b20de5a1145c6b8a11df8374e8790 in term 20.
W20260812 06:37:00.229668  9315 leader_election.cc:336] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 7962f587ccd842ad8a3f6ed9a5d7707e (127.4.177.3:40897): Network error: Client connection negotiation failed: client connection to 127.4.177.3:40897: connect: Connection refused (error 111)
I20260812 06:37:00.229754  9315 leader_election.cc:304] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 7 responses out of 7 voters: 3 yes votes; 4 no votes. yes voters: 222b20de5a1145c6b8a11df8374e8790, 8d90d61da62549208fe117b4384a469b, ec6c8d818e744cdbb6b0d15b321010e0; no voters: 2df4308c852240859ada53d216ab69ae, 42895a9bcba64aa484f6509605035d02, 7962f587ccd842ad8a3f6ed9a5d7707e, b037d17b21f8422fa5e41b70eb4dcd08
I20260812 06:37:00.229930 10447 raft_consensus.cc:2749] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260812 06:37:00.231640 10452 raft_consensus.cc:493] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 7962f587ccd842ad8a3f6ed9a5d7707e)
I20260812 06:37:00.231781 10452 raft_consensus.cc:515] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df4308c852240859ada53d216ab69ae" member_type: VOTER last_known_addr { host: "127.4.177.6" port: 34269 } } peers { permanent_uuid: "42895a9bcba64aa484f6509605035d02" member_type: VOTER last_known_addr { host: "127.4.177.4" port: 38181 } } peers { permanent_uuid: "222b20de5a1145c6b8a11df8374e8790" member_type: VOTER last_known_addr { host: "127.4.177.5" port: 43521 } } peers { permanent_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 40897 } } peers { permanent_uuid: "8d90d61da62549208fe117b4384a469b" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 41539 } } peers { permanent_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" member_type: VOTER last_known_addr { host: "127.4.177.7" port: 46299 } } peers { permanent_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 41057 } }
I20260812 06:37:00.232431 10452 leader_election.cc:290] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 2df4308c852240859ada53d216ab69ae (127.4.177.6:34269), 42895a9bcba64aa484f6509605035d02 (127.4.177.4:38181), 222b20de5a1145c6b8a11df8374e8790 (127.4.177.5:43521), 7962f587ccd842ad8a3f6ed9a5d7707e (127.4.177.3:40897), 8d90d61da62549208fe117b4384a469b (127.4.177.2:41539), ec6c8d818e744cdbb6b0d15b321010e0 (127.4.177.1:41057)
I20260812 06:37:00.232674  9797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" candidate_term: 21 candidate_status { last_received { term: 20 index: 2931 } } ignore_live_leader: false dest_uuid: "2df4308c852240859ada53d216ab69ae" is_pre_election: true
I20260812 06:37:00.232762  6686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" candidate_term: 21 candidate_status { last_received { term: 20 index: 2931 } } ignore_live_leader: false dest_uuid: "42895a9bcba64aa484f6509605035d02" is_pre_election: true
I20260812 06:37:00.232836  9797 raft_consensus.cc:2410] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b037d17b21f8422fa5e41b70eb4dcd08 for term 21 because replica has last-logged OpId of term: 20 index: 3028, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 2931.
I20260812 06:37:00.232864  9421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" candidate_term: 21 candidate_status { last_received { term: 20 index: 2931 } } ignore_live_leader: false dest_uuid: "222b20de5a1145c6b8a11df8374e8790" is_pre_election: true
I20260812 06:37:00.232926  6686 raft_consensus.cc:2410] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b037d17b21f8422fa5e41b70eb4dcd08 for term 21 because replica has last-logged OpId of term: 20 index: 3028, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 2931.
I20260812 06:37:00.233006  9421 raft_consensus.cc:2410] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b037d17b21f8422fa5e41b70eb4dcd08 for term 21 because replica has last-logged OpId of term: 20 index: 3020, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 2931.
W20260812 06:37:00.233457  7211 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.4.177.3:40897: connect: Connection refused (error 111) [suppressed 1 similar messages]
I20260812 06:37:00.233434  6048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" candidate_term: 21 candidate_status { last_received { term: 20 index: 2931 } } ignore_live_leader: false dest_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" is_pre_election: true
I20260812 06:37:00.233536  6241 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" candidate_term: 21 candidate_status { last_received { term: 20 index: 2931 } } ignore_live_leader: false dest_uuid: "8d90d61da62549208fe117b4384a469b" is_pre_election: true
I20260812 06:37:00.233600  6048 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P ec6c8d818e744cdbb6b0d15b321010e0 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b037d17b21f8422fa5e41b70eb4dcd08 in term 20.
I20260812 06:37:00.233700  6241 raft_consensus.cc:2410] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b037d17b21f8422fa5e41b70eb4dcd08 for term 21 because replica has last-logged OpId of term: 20 index: 2971, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 2931.
I20260812 06:37:00.234210  7211 leader_election.cc:304] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 6 responses out of 7 voters: 2 yes votes; 4 no votes. yes voters: b037d17b21f8422fa5e41b70eb4dcd08, ec6c8d818e744cdbb6b0d15b321010e0; no voters: 222b20de5a1145c6b8a11df8374e8790, 2df4308c852240859ada53d216ab69ae, 42895a9bcba64aa484f6509605035d02, 8d90d61da62549208fe117b4384a469b
W20260812 06:37:00.234403  7211 leader_election.cc:336] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 7962f587ccd842ad8a3f6ed9a5d7707e (127.4.177.3:40897): Network error: Client connection negotiation failed: client connection to 127.4.177.3:40897: connect: Connection refused (error 111)
I20260812 06:37:00.234551 10452 raft_consensus.cc:2749] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260812 06:37:01.630180 10458 raft_consensus.cc:493] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 7962f587ccd842ad8a3f6ed9a5d7707e)
I20260812 06:37:01.630411 10458 raft_consensus.cc:515] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df4308c852240859ada53d216ab69ae" member_type: VOTER last_known_addr { host: "127.4.177.6" port: 34269 } } peers { permanent_uuid: "42895a9bcba64aa484f6509605035d02" member_type: VOTER last_known_addr { host: "127.4.177.4" port: 38181 } } peers { permanent_uuid: "222b20de5a1145c6b8a11df8374e8790" member_type: VOTER last_known_addr { host: "127.4.177.5" port: 43521 } } peers { permanent_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 40897 } } peers { permanent_uuid: "8d90d61da62549208fe117b4384a469b" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 41539 } } peers { permanent_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" member_type: VOTER last_known_addr { host: "127.4.177.7" port: 46299 } } peers { permanent_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 41057 } }
I20260812 06:37:01.631130 10458 leader_election.cc:290] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 2df4308c852240859ada53d216ab69ae (127.4.177.6:34269), 222b20de5a1145c6b8a11df8374e8790 (127.4.177.5:43521), 7962f587ccd842ad8a3f6ed9a5d7707e (127.4.177.3:40897), 8d90d61da62549208fe117b4384a469b (127.4.177.2:41539), b037d17b21f8422fa5e41b70eb4dcd08 (127.4.177.7:46299), ec6c8d818e744cdbb6b0d15b321010e0 (127.4.177.1:41057)
I20260812 06:37:01.631603  9797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "42895a9bcba64aa484f6509605035d02" candidate_term: 21 candidate_status { last_received { term: 20 index: 3028 } } ignore_live_leader: false dest_uuid: "2df4308c852240859ada53d216ab69ae" is_pre_election: true
I20260812 06:37:01.631603  7320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "42895a9bcba64aa484f6509605035d02" candidate_term: 21 candidate_status { last_received { term: 20 index: 3028 } } ignore_live_leader: false dest_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" is_pre_election: true
I20260812 06:37:01.631754  7320 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 42895a9bcba64aa484f6509605035d02 in term 20.
I20260812 06:37:01.631754  9797 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 42895a9bcba64aa484f6509605035d02 in term 20.
I20260812 06:37:01.631739  9421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "42895a9bcba64aa484f6509605035d02" candidate_term: 21 candidate_status { last_received { term: 20 index: 3028 } } ignore_live_leader: false dest_uuid: "222b20de5a1145c6b8a11df8374e8790" is_pre_election: true
I20260812 06:37:01.631942  9421 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 42895a9bcba64aa484f6509605035d02 in term 20.
I20260812 06:37:01.632196  6241 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "42895a9bcba64aa484f6509605035d02" candidate_term: 21 candidate_status { last_received { term: 20 index: 3028 } } ignore_live_leader: false dest_uuid: "8d90d61da62549208fe117b4384a469b" is_pre_election: true
I20260812 06:37:01.632354  6241 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 42895a9bcba64aa484f6509605035d02 in term 20.
I20260812 06:37:01.632386  6048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "42895a9bcba64aa484f6509605035d02" candidate_term: 21 candidate_status { last_received { term: 20 index: 3028 } } ignore_live_leader: false dest_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" is_pre_election: true
I20260812 06:37:01.632547  6048 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P ec6c8d818e744cdbb6b0d15b321010e0 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 42895a9bcba64aa484f6509605035d02 in term 20.
I20260812 06:37:01.632700  6597 leader_election.cc:304] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 7 voters: 4 yes votes; 0 no votes. yes voters: 222b20de5a1145c6b8a11df8374e8790, 2df4308c852240859ada53d216ab69ae, 42895a9bcba64aa484f6509605035d02, b037d17b21f8422fa5e41b70eb4dcd08; no voters: 
I20260812 06:37:01.632910 10458 raft_consensus.cc:2804] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260812 06:37:01.632980 10458 raft_consensus.cc:493] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 7962f587ccd842ad8a3f6ed9a5d7707e)
I20260812 06:37:01.633033 10458 raft_consensus.cc:3060] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [term 20 FOLLOWER]: Advancing to term 21
W20260812 06:37:01.633378  6597 leader_election.cc:336] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 7962f587ccd842ad8a3f6ed9a5d7707e (127.4.177.3:40897): Network error: Client connection negotiation failed: client connection to 127.4.177.3:40897: connect: Connection refused (error 111)
I20260812 06:37:01.634130 10458 raft_consensus.cc:515] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df4308c852240859ada53d216ab69ae" member_type: VOTER last_known_addr { host: "127.4.177.6" port: 34269 } } peers { permanent_uuid: "42895a9bcba64aa484f6509605035d02" member_type: VOTER last_known_addr { host: "127.4.177.4" port: 38181 } } peers { permanent_uuid: "222b20de5a1145c6b8a11df8374e8790" member_type: VOTER last_known_addr { host: "127.4.177.5" port: 43521 } } peers { permanent_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 40897 } } peers { permanent_uuid: "8d90d61da62549208fe117b4384a469b" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 41539 } } peers { permanent_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" member_type: VOTER last_known_addr { host: "127.4.177.7" port: 46299 } } peers { permanent_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 41057 } }
I20260812 06:37:01.634811 10458 leader_election.cc:290] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [CANDIDATE]: Term 21 election: Requested vote from peers 2df4308c852240859ada53d216ab69ae (127.4.177.6:34269), 222b20de5a1145c6b8a11df8374e8790 (127.4.177.5:43521), 7962f587ccd842ad8a3f6ed9a5d7707e (127.4.177.3:40897), 8d90d61da62549208fe117b4384a469b (127.4.177.2:41539), b037d17b21f8422fa5e41b70eb4dcd08 (127.4.177.7:46299), ec6c8d818e744cdbb6b0d15b321010e0 (127.4.177.1:41057)
I20260812 06:37:01.634917  9797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "42895a9bcba64aa484f6509605035d02" candidate_term: 21 candidate_status { last_received { term: 20 index: 3028 } } ignore_live_leader: false dest_uuid: "2df4308c852240859ada53d216ab69ae"
I20260812 06:37:01.635057  9797 raft_consensus.cc:3060] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [term 20 FOLLOWER]: Advancing to term 21
I20260812 06:37:01.635134  7320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "42895a9bcba64aa484f6509605035d02" candidate_term: 21 candidate_status { last_received { term: 20 index: 3028 } } ignore_live_leader: false dest_uuid: "b037d17b21f8422fa5e41b70eb4dcd08"
I20260812 06:37:01.635249  7320 raft_consensus.cc:3060] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08 [term 20 FOLLOWER]: Advancing to term 21
I20260812 06:37:01.635237  9421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "42895a9bcba64aa484f6509605035d02" candidate_term: 21 candidate_status { last_received { term: 20 index: 3028 } } ignore_live_leader: false dest_uuid: "222b20de5a1145c6b8a11df8374e8790"
I20260812 06:37:01.635293  6241 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "42895a9bcba64aa484f6509605035d02" candidate_term: 21 candidate_status { last_received { term: 20 index: 3028 } } ignore_live_leader: false dest_uuid: "8d90d61da62549208fe117b4384a469b"
I20260812 06:37:01.635360  9421 raft_consensus.cc:3060] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [term 20 FOLLOWER]: Advancing to term 21
I20260812 06:37:01.635411  6241 raft_consensus.cc:3060] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b [term 20 FOLLOWER]: Advancing to term 21
I20260812 06:37:01.636161  9797 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 42895a9bcba64aa484f6509605035d02 in term 21.
I20260812 06:37:01.636325  7320 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 42895a9bcba64aa484f6509605035d02 in term 21.
I20260812 06:37:01.636404  9421 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 42895a9bcba64aa484f6509605035d02 in term 21.
I20260812 06:37:01.636404  6241 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 42895a9bcba64aa484f6509605035d02 in term 21.
I20260812 06:37:01.636936  6048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "42895a9bcba64aa484f6509605035d02" candidate_term: 21 candidate_status { last_received { term: 20 index: 3028 } } ignore_live_leader: false dest_uuid: "ec6c8d818e744cdbb6b0d15b321010e0"
I20260812 06:37:01.637080  6048 raft_consensus.cc:3060] T 7065bb403e3047dd9a0853c49318ef1f P ec6c8d818e744cdbb6b0d15b321010e0 [term 20 FOLLOWER]: Advancing to term 21
I20260812 06:37:01.637121  6597 leader_election.cc:304] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 7 voters: 4 yes votes; 0 no votes. yes voters: 222b20de5a1145c6b8a11df8374e8790, 42895a9bcba64aa484f6509605035d02, 8d90d61da62549208fe117b4384a469b, b037d17b21f8422fa5e41b70eb4dcd08; no voters: 
I20260812 06:37:01.637282 10458 raft_consensus.cc:2804] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [term 21 FOLLOWER]: Leader election won for term 21
I20260812 06:37:01.637491 10458 raft_consensus.cc:697] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [term 21 LEADER]: Becoming Leader. State: Replica: 42895a9bcba64aa484f6509605035d02, State: Running, Role: LEADER
I20260812 06:37:01.637709 10458 consensus_queue.cc:237] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3020, Committed index: 3020, Last appended: 20.3028, Last appended by leader: 3028, Current term: 21, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df4308c852240859ada53d216ab69ae" member_type: VOTER last_known_addr { host: "127.4.177.6" port: 34269 } } peers { permanent_uuid: "42895a9bcba64aa484f6509605035d02" member_type: VOTER last_known_addr { host: "127.4.177.4" port: 38181 } } peers { permanent_uuid: "222b20de5a1145c6b8a11df8374e8790" member_type: VOTER last_known_addr { host: "127.4.177.5" port: 43521 } } peers { permanent_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 40897 } } peers { permanent_uuid: "8d90d61da62549208fe117b4384a469b" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 41539 } } peers { permanent_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" member_type: VOTER last_known_addr { host: "127.4.177.7" port: 46299 } } peers { permanent_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 41057 } }
W20260812 06:37:01.637907  6597 leader_election.cc:336] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer 7962f587ccd842ad8a3f6ed9a5d7707e (127.4.177.3:40897): Network error: Client connection negotiation failed: client connection to 127.4.177.3:40897: connect: Connection refused (error 111)
I20260812 06:37:01.638303  6048 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P ec6c8d818e744cdbb6b0d15b321010e0 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 42895a9bcba64aa484f6509605035d02 in term 21.
I20260812 06:37:01.640247  5808 catalog_manager.cc:5719] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 reported cstate change: term changed from 20 to 21, leader changed from 7962f587ccd842ad8a3f6ed9a5d7707e (127.4.177.3) to 42895a9bcba64aa484f6509605035d02 (127.4.177.4). New cstate: current_term: 21 leader_uuid: "42895a9bcba64aa484f6509605035d02" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df4308c852240859ada53d216ab69ae" member_type: VOTER last_known_addr { host: "127.4.177.6" port: 34269 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "42895a9bcba64aa484f6509605035d02" member_type: VOTER last_known_addr { host: "127.4.177.4" port: 38181 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "222b20de5a1145c6b8a11df8374e8790" member_type: VOTER last_known_addr { host: "127.4.177.5" port: 43521 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 40897 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8d90d61da62549208fe117b4384a469b" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 41539 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" member_type: VOTER last_known_addr { host: "127.4.177.7" port: 46299 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 41057 } health_report { overall_health: UNKNOWN } } }
I20260812 06:37:02.033778  7320 raft_consensus.cc:1275] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08 [term 21 FOLLOWER]: Refusing update from remote peer 42895a9bcba64aa484f6509605035d02: Log matching property violated. Preceding OpId in replica: term: 20 index: 2931. Preceding OpId from leader: term: 21 index: 3029. (index mismatch)
I20260812 06:37:02.034363 10458 consensus_queue.cc:1048] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" member_type: VOTER last_known_addr { host: "127.4.177.7" port: 46299 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3029, Last known committed idx: 2926, Time since last communication: 0.000s
I20260812 06:37:02.096043  9421 raft_consensus.cc:1275] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [term 21 FOLLOWER]: Refusing update from remote peer 42895a9bcba64aa484f6509605035d02: Log matching property violated. Preceding OpId in replica: term: 20 index: 3020. Preceding OpId from leader: term: 21 index: 3029. (index mismatch)
I20260812 06:37:02.096707 10458 consensus_queue.cc:1048] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [LEADER]: Connected to new peer: Peer: permanent_uuid: "222b20de5a1145c6b8a11df8374e8790" member_type: VOTER last_known_addr { host: "127.4.177.5" port: 43521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3029, Last known committed idx: 3016, Time since last communication: 0.000s
W20260812 06:37:02.116056  6597 consensus_peers.cc:597] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 -> Peer 7962f587ccd842ad8a3f6ed9a5d7707e (127.4.177.3:40897): Couldn't send request to peer 7962f587ccd842ad8a3f6ed9a5d7707e. Status: Network error: Client connection negotiation failed: client connection to 127.4.177.3:40897: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260812 06:37:02.119175  6048 raft_consensus.cc:1275] T 7065bb403e3047dd9a0853c49318ef1f P ec6c8d818e744cdbb6b0d15b321010e0 [term 21 FOLLOWER]: Refusing update from remote peer 42895a9bcba64aa484f6509605035d02: Log matching property violated. Preceding OpId in replica: term: 20 index: 2882. Preceding OpId from leader: term: 21 index: 3029. (index mismatch)
I20260812 06:37:02.119737 10458 consensus_queue.cc:1048] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 41057 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3029, Last known committed idx: 2878, Time since last communication: 0.000s
I20260812 06:37:02.143643  6241 raft_consensus.cc:1275] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b [term 21 FOLLOWER]: Refusing update from remote peer 42895a9bcba64aa484f6509605035d02: Log matching property violated. Preceding OpId in replica: term: 20 index: 2971. Preceding OpId from leader: term: 21 index: 3029. (index mismatch)
I20260812 06:37:02.144248 10458 consensus_queue.cc:1048] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8d90d61da62549208fe117b4384a469b" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 41539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3029, Last known committed idx: 2968, Time since last communication: 0.000s
I20260812 06:37:02.161919  9797 raft_consensus.cc:1275] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [term 21 FOLLOWER]: Refusing update from remote peer 42895a9bcba64aa484f6509605035d02: Log matching property violated. Preceding OpId in replica: term: 20 index: 3028. Preceding OpId from leader: term: 21 index: 3029. (index mismatch)
I20260812 06:37:02.166891 10458 consensus_queue.cc:1048] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2df4308c852240859ada53d216ab69ae" member_type: VOTER last_known_addr { host: "127.4.177.6" port: 34269 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3029, Last known committed idx: 3020, Time since last communication: 0.000s
I20260812 06:37:02.182019 10540 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:41528 (ReqId={client: 3067f3aef2a44cc4bb7f2ccd6acf83db, seq_no=3005, attempt_no=1}) took 3452 ms. Trace:
I20260812 06:37:02.182257 10540 rpcz_store.cc:276] 0812 06:36:58.730060 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 06:36:58.733024 (+  2964us) service_pool.cc:224] Handling call
0812 06:37:02.181997 (+3448973us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260812 06:37:02.183171 10540 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:41528 (ReqId={client: 3067f3aef2a44cc4bb7f2ccd6acf83db, seq_no=3006, attempt_no=1}) took 3453 ms. Trace:
I20260812 06:37:02.183317 10540 rpcz_store.cc:276] 0812 06:36:58.730136 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 06:36:58.731273 (+  1137us) service_pool.cc:224] Handling call
0812 06:37:02.183157 (+3451884us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260812 06:37:02.184760  9744 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:41528: Illegal state: replica 2df4308c852240859ada53d216ab69ae is not leader of this config: current role FOLLOWER
W20260812 06:37:02.185784  9744 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:41528: Illegal state: replica 2df4308c852240859ada53d216ab69ae is not leader of this config: current role FOLLOWER
I20260812 06:37:02.186734 10540 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:41528 (ReqId={client: 3067f3aef2a44cc4bb7f2ccd6acf83db, seq_no=3007, attempt_no=1}) took 3456 ms. Trace:
I20260812 06:37:02.186862 10540 rpcz_store.cc:276] 0812 06:36:58.730195 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 06:36:58.730958 (+   763us) service_pool.cc:224] Handling call
0812 06:37:02.186719 (+3455761us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260812 06:37:02.189041 10540 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:41528 (ReqId={client: 3067f3aef2a44cc4bb7f2ccd6acf83db, seq_no=3008, attempt_no=1}) took 3458 ms. Trace:
I20260812 06:37:02.189178 10540 rpcz_store.cc:276] 0812 06:36:58.730276 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 06:36:58.731219 (+   943us) service_pool.cc:224] Handling call
0812 06:37:02.189022 (+3457803us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260812 06:37:02.189348 10544 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:41528 (ReqId={client: 3067f3aef2a44cc4bb7f2ccd6acf83db, seq_no=3009, attempt_no=1}) took 3458 ms. Trace:
I20260812 06:37:02.189416 10544 rpcz_store.cc:276] 0812 06:36:58.730392 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 06:36:58.731385 (+   993us) service_pool.cc:224] Handling call
0812 06:37:02.189334 (+3457949us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260812 06:37:02.190852 10543 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:41528 (ReqId={client: 3067f3aef2a44cc4bb7f2ccd6acf83db, seq_no=3010, attempt_no=1}) took 3460 ms. Trace:
I20260812 06:37:02.190928 10543 rpcz_store.cc:276] 0812 06:36:58.730440 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 06:36:58.731326 (+   886us) service_pool.cc:224] Handling call
0812 06:37:02.190838 (+3459512us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260812 06:37:02.191592 10543 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:41528 (ReqId={client: 3067f3aef2a44cc4bb7f2ccd6acf83db, seq_no=3011, attempt_no=1}) took 3461 ms. Trace:
I20260812 06:37:02.196770 10543 rpcz_store.cc:276] 0812 06:36:58.730486 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 06:36:58.731150 (+   664us) service_pool.cc:224] Handling call
0812 06:37:02.191579 (+3460429us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260812 06:37:02.192487 10540 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:41528 (ReqId={client: 3067f3aef2a44cc4bb7f2ccd6acf83db, seq_no=3012, attempt_no=1}) took 3462 ms. Trace:
I20260812 06:37:02.197191 10540 rpcz_store.cc:276] 0812 06:36:58.730338 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 06:36:58.731083 (+   745us) service_pool.cc:224] Handling call
0812 06:37:02.192474 (+3461391us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260812 06:37:02.421772  4804 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskCnZu03/build/debug/bin/kudu
/tmp/dist-test-taskCnZu03/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.177.3:40897
--local_ip_for_outbound_sockets=127.4.177.3
--tserver_master_addrs=127.4.177.62:42183
--webserver_port=44527
--webserver_interface=127.4.177.3
--builtin_ntp_servers=127.4.177.20:34335
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_cache_size_limit_mb=10
--consensus_rpc_timeout_ms=1000 with env {}
W20260812 06:37:02.684561 10556 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:37:02.684949 10556 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:37:02.685061 10556 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 06:37:02.692466 10556 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:37:02.692682 10556 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.177.3
I20260812 06:37:02.700062 10556 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-2/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.177.20:34335
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--consensus_rpc_timeout_ms=1000
--log_cache_size_limit_mb=10
--fs_data_dirs=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.177.3:40897
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--webserver_interface=127.4.177.3
--webserver_port=44527
--tserver_master_addrs=127.4.177.62:42183
--never_fsync=true
--heap_profile_path=/tmp/kudu.10556
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.177.3

Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 06:37:02.706535 10556 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:37:02.708204 10556 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 06:37:02.725054 10573 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:37:02.725715 10569 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:37:02.728260 10568 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:37:02.729264 10556 server_base.cc:1061] running on GCE node
I20260812 06:37:02.730978 10556 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:37:02.731732 10556 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:37:02.739084 10556 hybrid_clock.cc:648] HybridClock initialized: now 1786516622738860 us; error 234 us; skew 500 ppm
I20260812 06:37:02.742540 10556 webserver.cc:533] Webserver started at http://127.4.177.3:44527/ using document root <none> and password file <none>
I20260812 06:37:02.743418 10556 fs_manager.cc:362] Metadata directory not provided
I20260812 06:37:02.743567 10556 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:37:02.762763 10556 fs_manager.cc:714] Time spent opening directory manager: real 0.017s	user 0.004s	sys 0.000s
I20260812 06:37:02.768235 10582 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 06:37:02.769582 10556 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.002s	sys 0.000s
I20260812 06:37:02.769708 10556 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-2/data,/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-2/wal
uuid: "7962f587ccd842ad8a3f6ed9a5d7707e"
format_stamp: "Formatted at 2026-08-12 06:36:11 on dist-test-slave-jztv"
I20260812 06:37:02.770272 10556 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-2/wal
metadata directory: /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-2/wal
1 data directories: /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:37:02.795825 10556 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 06:37:02.796778 10556 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:37:02.797056 10556 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:37:02.797824 10556 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260812 06:37:02.803076 10594 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
I20260812 06:37:02.807857 10556 ts_tablet_manager.cc:585] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260812 06:37:02.807920 10556 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.009s	user 0.001s	sys 0.000s
I20260812 06:37:02.807997 10556 ts_tablet_manager.cc:600] Registering tablets (0/1 complete)
I20260812 06:37:02.814786 10556 ts_tablet_manager.cc:616] Registered 1 tablets
I20260812 06:37:02.814867 10556 ts_tablet_manager.cc:595] Time spent register tablets: real 0.007s	user 0.002s	sys 0.000s
I20260812 06:37:02.830863 10594 tablet_bootstrap.cc:492] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e: Bootstrap starting.
I20260812 06:37:02.848129 10556 rpc_server.cc:307] RPC server started. Bound to: 127.4.177.3:40897
I20260812 06:37:02.849493 10556 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-2/data/info.pb
I20260812 06:37:02.852104 10768 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.4.177.3:40897 every 8 connection(s)
I20260812 06:37:02.853966  4804 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskCnZu03/build/debug/bin/kudu as pid 10556
I20260812 06:37:02.889691 10769 heartbeater.cc:344] Connected to a master server at 127.4.177.62:42183
I20260812 06:37:02.890020 10769 heartbeater.cc:461] Registering TS with master...
I20260812 06:37:02.890796 10769 heartbeater.cc:507] Master 127.4.177.62:42183 requested a full tablet report, sending...
I20260812 06:37:02.896622  5814 ts_manager.cc:194] Re-registered known tserver with Master: 7962f587ccd842ad8a3f6ed9a5d7707e (127.4.177.3:40897)
I20260812 06:37:02.897570  5814 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.4.177.3:49275
I20260812 06:37:03.373754 10594 log.cc:826] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e: Log is configured to *not* fsync() on all Append() calls
I20260812 06:37:03.907856 10769 heartbeater.cc:499] Master 127.4.177.62:42183 was elected leader, sending a full tablet report...
I20260812 06:37:04.774709 10786 raft_consensus.cc:493] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 42895a9bcba64aa484f6509605035d02)
I20260812 06:37:04.774849 10786 raft_consensus.cc:515] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df4308c852240859ada53d216ab69ae" member_type: VOTER last_known_addr { host: "127.4.177.6" port: 34269 } } peers { permanent_uuid: "42895a9bcba64aa484f6509605035d02" member_type: VOTER last_known_addr { host: "127.4.177.4" port: 38181 } } peers { permanent_uuid: "222b20de5a1145c6b8a11df8374e8790" member_type: VOTER last_known_addr { host: "127.4.177.5" port: 43521 } } peers { permanent_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 40897 } } peers { permanent_uuid: "8d90d61da62549208fe117b4384a469b" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 41539 } } peers { permanent_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" member_type: VOTER last_known_addr { host: "127.4.177.7" port: 46299 } } peers { permanent_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 41057 } }
I20260812 06:37:04.775655 10786 leader_election.cc:290] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 2df4308c852240859ada53d216ab69ae (127.4.177.6:34269), 42895a9bcba64aa484f6509605035d02 (127.4.177.4:38181), 222b20de5a1145c6b8a11df8374e8790 (127.4.177.5:43521), 7962f587ccd842ad8a3f6ed9a5d7707e (127.4.177.3:40897), 8d90d61da62549208fe117b4384a469b (127.4.177.2:41539), ec6c8d818e744cdbb6b0d15b321010e0 (127.4.177.1:41057)
I20260812 06:37:04.776223  9797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" candidate_term: 22 candidate_status { last_received { term: 21 index: 3295 } } ignore_live_leader: false dest_uuid: "2df4308c852240859ada53d216ab69ae" is_pre_election: true
I20260812 06:37:04.776535  9425 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" candidate_term: 22 candidate_status { last_received { term: 21 index: 3295 } } ignore_live_leader: false dest_uuid: "222b20de5a1145c6b8a11df8374e8790" is_pre_election: true
I20260812 06:37:04.776871  6048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" candidate_term: 22 candidate_status { last_received { term: 21 index: 3295 } } ignore_live_leader: false dest_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" is_pre_election: true
I20260812 06:37:04.776961  6236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" candidate_term: 22 candidate_status { last_received { term: 21 index: 3295 } } ignore_live_leader: false dest_uuid: "8d90d61da62549208fe117b4384a469b" is_pre_election: true
I20260812 06:37:04.777586  7211 leader_election.cc:304] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 5 responses out of 7 voters: 1 yes votes; 4 no votes. yes voters: b037d17b21f8422fa5e41b70eb4dcd08; no voters: 222b20de5a1145c6b8a11df8374e8790, 2df4308c852240859ada53d216ab69ae, 8d90d61da62549208fe117b4384a469b, ec6c8d818e744cdbb6b0d15b321010e0
I20260812 06:37:04.777787 10786 raft_consensus.cc:2749] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260812 06:37:04.781172 10694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" candidate_term: 22 candidate_status { last_received { term: 21 index: 3295 } } ignore_live_leader: false dest_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" is_pre_election: true
W20260812 06:37:04.782442  7211 leader_election.cc:343] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08 [CANDIDATE]: Term 22 pre-election: Tablet error from VoteRequest() call to peer 7962f587ccd842ad8a3f6ed9a5d7707e (127.4.177.3:40897): Illegal state: must be running to vote when last-logged opid is not known
I20260812 06:37:04.913059 10790 raft_consensus.cc:493] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 42895a9bcba64aa484f6509605035d02)
I20260812 06:37:04.913257 10790 raft_consensus.cc:515] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df4308c852240859ada53d216ab69ae" member_type: VOTER last_known_addr { host: "127.4.177.6" port: 34269 } } peers { permanent_uuid: "42895a9bcba64aa484f6509605035d02" member_type: VOTER last_known_addr { host: "127.4.177.4" port: 38181 } } peers { permanent_uuid: "222b20de5a1145c6b8a11df8374e8790" member_type: VOTER last_known_addr { host: "127.4.177.5" port: 43521 } } peers { permanent_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 40897 } } peers { permanent_uuid: "8d90d61da62549208fe117b4384a469b" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 41539 } } peers { permanent_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" member_type: VOTER last_known_addr { host: "127.4.177.7" port: 46299 } } peers { permanent_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 41057 } }
I20260812 06:37:04.913367 10792 raft_consensus.cc:493] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 42895a9bcba64aa484f6509605035d02)
I20260812 06:37:04.913552 10792 raft_consensus.cc:515] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df4308c852240859ada53d216ab69ae" member_type: VOTER last_known_addr { host: "127.4.177.6" port: 34269 } } peers { permanent_uuid: "42895a9bcba64aa484f6509605035d02" member_type: VOTER last_known_addr { host: "127.4.177.4" port: 38181 } } peers { permanent_uuid: "222b20de5a1145c6b8a11df8374e8790" member_type: VOTER last_known_addr { host: "127.4.177.5" port: 43521 } } peers { permanent_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 40897 } } peers { permanent_uuid: "8d90d61da62549208fe117b4384a469b" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 41539 } } peers { permanent_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" member_type: VOTER last_known_addr { host: "127.4.177.7" port: 46299 } } peers { permanent_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 41057 } }
I20260812 06:37:04.914152 10790 leader_election.cc:290] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 42895a9bcba64aa484f6509605035d02 (127.4.177.4:38181), 222b20de5a1145c6b8a11df8374e8790 (127.4.177.5:43521), 7962f587ccd842ad8a3f6ed9a5d7707e (127.4.177.3:40897), 8d90d61da62549208fe117b4384a469b (127.4.177.2:41539), b037d17b21f8422fa5e41b70eb4dcd08 (127.4.177.7:46299), ec6c8d818e744cdbb6b0d15b321010e0 (127.4.177.1:41057)
I20260812 06:37:04.914347  7322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "2df4308c852240859ada53d216ab69ae" candidate_term: 22 candidate_status { last_received { term: 21 index: 3338 } } ignore_live_leader: false dest_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" is_pre_election: true
I20260812 06:37:04.914505  7322 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2df4308c852240859ada53d216ab69ae in term 21.
I20260812 06:37:04.914703  9425 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "2df4308c852240859ada53d216ab69ae" candidate_term: 22 candidate_status { last_received { term: 21 index: 3338 } } ignore_live_leader: false dest_uuid: "222b20de5a1145c6b8a11df8374e8790" is_pre_election: true
I20260812 06:37:04.915068  9425 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2df4308c852240859ada53d216ab69ae in term 21.
I20260812 06:37:04.915139  6048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "2df4308c852240859ada53d216ab69ae" candidate_term: 22 candidate_status { last_received { term: 21 index: 3338 } } ignore_live_leader: false dest_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" is_pre_election: true
I20260812 06:37:04.915700  6236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "2df4308c852240859ada53d216ab69ae" candidate_term: 22 candidate_status { last_received { term: 21 index: 3338 } } ignore_live_leader: false dest_uuid: "8d90d61da62549208fe117b4384a469b" is_pre_election: true
I20260812 06:37:04.916003  9797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "222b20de5a1145c6b8a11df8374e8790" candidate_term: 22 candidate_status { last_received { term: 21 index: 3338 } } ignore_live_leader: false dest_uuid: "2df4308c852240859ada53d216ab69ae" is_pre_election: true
I20260812 06:37:04.916132  9797 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 222b20de5a1145c6b8a11df8374e8790 in term 21.
I20260812 06:37:04.916597 10792 leader_election.cc:290] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 2df4308c852240859ada53d216ab69ae (127.4.177.6:34269), 42895a9bcba64aa484f6509605035d02 (127.4.177.4:38181), 7962f587ccd842ad8a3f6ed9a5d7707e (127.4.177.3:40897), 8d90d61da62549208fe117b4384a469b (127.4.177.2:41539), b037d17b21f8422fa5e41b70eb4dcd08 (127.4.177.7:46299), ec6c8d818e744cdbb6b0d15b321010e0 (127.4.177.1:41057)
I20260812 06:37:04.917168  6236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "222b20de5a1145c6b8a11df8374e8790" candidate_term: 22 candidate_status { last_received { term: 21 index: 3338 } } ignore_live_leader: false dest_uuid: "8d90d61da62549208fe117b4384a469b" is_pre_election: true
I20260812 06:37:04.917168  7322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "222b20de5a1145c6b8a11df8374e8790" candidate_term: 22 candidate_status { last_received { term: 21 index: 3338 } } ignore_live_leader: false dest_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" is_pre_election: true
I20260812 06:37:04.917280  7322 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 222b20de5a1145c6b8a11df8374e8790 in term 21.
I20260812 06:37:04.917508  6048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "222b20de5a1145c6b8a11df8374e8790" candidate_term: 22 candidate_status { last_received { term: 21 index: 3338 } } ignore_live_leader: false dest_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" is_pre_election: true
I20260812 06:37:04.920712 10694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "2df4308c852240859ada53d216ab69ae" candidate_term: 22 candidate_status { last_received { term: 21 index: 3338 } } ignore_live_leader: false dest_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" is_pre_election: true
W20260812 06:37:04.921121  9714 leader_election.cc:343] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [CANDIDATE]: Term 22 pre-election: Tablet error from VoteRequest() call to peer 7962f587ccd842ad8a3f6ed9a5d7707e (127.4.177.3:40897): Illegal state: must be running to vote when last-logged opid is not known
I20260812 06:37:04.921789 10694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "222b20de5a1145c6b8a11df8374e8790" candidate_term: 22 candidate_status { last_received { term: 21 index: 3338 } } ignore_live_leader: false dest_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" is_pre_election: true
W20260812 06:37:04.922155  9315 leader_election.cc:343] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [CANDIDATE]: Term 22 pre-election: Tablet error from VoteRequest() call to peer 7962f587ccd842ad8a3f6ed9a5d7707e (127.4.177.3:40897): Illegal state: must be running to vote when last-logged opid is not known
I20260812 06:37:05.350675 10800 raft_consensus.cc:493] T 7065bb403e3047dd9a0853c49318ef1f P ec6c8d818e744cdbb6b0d15b321010e0 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 42895a9bcba64aa484f6509605035d02)
I20260812 06:37:05.350909 10800 raft_consensus.cc:515] T 7065bb403e3047dd9a0853c49318ef1f P ec6c8d818e744cdbb6b0d15b321010e0 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df4308c852240859ada53d216ab69ae" member_type: VOTER last_known_addr { host: "127.4.177.6" port: 34269 } } peers { permanent_uuid: "42895a9bcba64aa484f6509605035d02" member_type: VOTER last_known_addr { host: "127.4.177.4" port: 38181 } } peers { permanent_uuid: "222b20de5a1145c6b8a11df8374e8790" member_type: VOTER last_known_addr { host: "127.4.177.5" port: 43521 } } peers { permanent_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 40897 } } peers { permanent_uuid: "8d90d61da62549208fe117b4384a469b" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 41539 } } peers { permanent_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" member_type: VOTER last_known_addr { host: "127.4.177.7" port: 46299 } } peers { permanent_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 41057 } }
I20260812 06:37:05.351838 10800 leader_election.cc:290] T 7065bb403e3047dd9a0853c49318ef1f P ec6c8d818e744cdbb6b0d15b321010e0 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 2df4308c852240859ada53d216ab69ae (127.4.177.6:34269), 42895a9bcba64aa484f6509605035d02 (127.4.177.4:38181), 222b20de5a1145c6b8a11df8374e8790 (127.4.177.5:43521), 7962f587ccd842ad8a3f6ed9a5d7707e (127.4.177.3:40897), 8d90d61da62549208fe117b4384a469b (127.4.177.2:41539), b037d17b21f8422fa5e41b70eb4dcd08 (127.4.177.7:46299)
I20260812 06:37:05.352175  9797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" candidate_term: 22 candidate_status { last_received { term: 21 index: 3272 } } ignore_live_leader: false dest_uuid: "2df4308c852240859ada53d216ab69ae" is_pre_election: true
I20260812 06:37:05.352377  9797 raft_consensus.cc:2410] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ec6c8d818e744cdbb6b0d15b321010e0 for term 22 because replica has last-logged OpId of term: 21 index: 3338, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 3272.
I20260812 06:37:05.352468  7322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" candidate_term: 22 candidate_status { last_received { term: 21 index: 3272 } } ignore_live_leader: false dest_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" is_pre_election: true
I20260812 06:37:05.352644  7322 raft_consensus.cc:2410] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ec6c8d818e744cdbb6b0d15b321010e0 for term 22 because replica has last-logged OpId of term: 21 index: 3295, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 3272.
I20260812 06:37:05.352880  6236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" candidate_term: 22 candidate_status { last_received { term: 21 index: 3272 } } ignore_live_leader: false dest_uuid: "8d90d61da62549208fe117b4384a469b" is_pre_election: true
I20260812 06:37:05.353303  9425 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" candidate_term: 22 candidate_status { last_received { term: 21 index: 3272 } } ignore_live_leader: false dest_uuid: "222b20de5a1145c6b8a11df8374e8790" is_pre_election: true
I20260812 06:37:05.353506  9425 raft_consensus.cc:2410] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ec6c8d818e744cdbb6b0d15b321010e0 for term 22 because replica has last-logged OpId of term: 21 index: 3338, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 3272.
I20260812 06:37:05.353866  5945 leader_election.cc:304] T 7065bb403e3047dd9a0853c49318ef1f P ec6c8d818e744cdbb6b0d15b321010e0 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 5 responses out of 7 voters: 1 yes votes; 4 no votes. yes voters: ec6c8d818e744cdbb6b0d15b321010e0; no voters: 222b20de5a1145c6b8a11df8374e8790, 2df4308c852240859ada53d216ab69ae, 8d90d61da62549208fe117b4384a469b, b037d17b21f8422fa5e41b70eb4dcd08
I20260812 06:37:05.354061 10800 raft_consensus.cc:2749] T 7065bb403e3047dd9a0853c49318ef1f P ec6c8d818e744cdbb6b0d15b321010e0 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260812 06:37:05.356709 10803 raft_consensus.cc:493] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 42895a9bcba64aa484f6509605035d02)
I20260812 06:37:05.356853 10694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" candidate_term: 22 candidate_status { last_received { term: 21 index: 3272 } } ignore_live_leader: false dest_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" is_pre_election: true
I20260812 06:37:05.356843 10803 raft_consensus.cc:515] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df4308c852240859ada53d216ab69ae" member_type: VOTER last_known_addr { host: "127.4.177.6" port: 34269 } } peers { permanent_uuid: "42895a9bcba64aa484f6509605035d02" member_type: VOTER last_known_addr { host: "127.4.177.4" port: 38181 } } peers { permanent_uuid: "222b20de5a1145c6b8a11df8374e8790" member_type: VOTER last_known_addr { host: "127.4.177.5" port: 43521 } } peers { permanent_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 40897 } } peers { permanent_uuid: "8d90d61da62549208fe117b4384a469b" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 41539 } } peers { permanent_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" member_type: VOTER last_known_addr { host: "127.4.177.7" port: 46299 } } peers { permanent_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 41057 } }
W20260812 06:37:05.357239  5945 leader_election.cc:343] T 7065bb403e3047dd9a0853c49318ef1f P ec6c8d818e744cdbb6b0d15b321010e0 [CANDIDATE]: Term 22 pre-election: Tablet error from VoteRequest() call to peer 7962f587ccd842ad8a3f6ed9a5d7707e (127.4.177.3:40897): Illegal state: must be running to vote when last-logged opid is not known
I20260812 06:37:05.357770 10803 leader_election.cc:290] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 2df4308c852240859ada53d216ab69ae (127.4.177.6:34269), 42895a9bcba64aa484f6509605035d02 (127.4.177.4:38181), 222b20de5a1145c6b8a11df8374e8790 (127.4.177.5:43521), 7962f587ccd842ad8a3f6ed9a5d7707e (127.4.177.3:40897), b037d17b21f8422fa5e41b70eb4dcd08 (127.4.177.7:46299), ec6c8d818e744cdbb6b0d15b321010e0 (127.4.177.1:41057)
I20260812 06:37:05.358016  9797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "8d90d61da62549208fe117b4384a469b" candidate_term: 22 candidate_status { last_received { term: 21 index: 3331 } } ignore_live_leader: false dest_uuid: "2df4308c852240859ada53d216ab69ae" is_pre_election: true
I20260812 06:37:05.358204  9797 raft_consensus.cc:2410] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8d90d61da62549208fe117b4384a469b for term 22 because replica has last-logged OpId of term: 21 index: 3338, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 3331.
I20260812 06:37:05.358672  9425 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "8d90d61da62549208fe117b4384a469b" candidate_term: 22 candidate_status { last_received { term: 21 index: 3331 } } ignore_live_leader: false dest_uuid: "222b20de5a1145c6b8a11df8374e8790" is_pre_election: true
I20260812 06:37:05.358850  9425 raft_consensus.cc:2410] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8d90d61da62549208fe117b4384a469b for term 22 because replica has last-logged OpId of term: 21 index: 3338, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 3331.
I20260812 06:37:05.359063  6048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "8d90d61da62549208fe117b4384a469b" candidate_term: 22 candidate_status { last_received { term: 21 index: 3331 } } ignore_live_leader: false dest_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" is_pre_election: true
I20260812 06:37:05.359220  6048 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P ec6c8d818e744cdbb6b0d15b321010e0 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8d90d61da62549208fe117b4384a469b in term 21.
I20260812 06:37:05.359246  7322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "8d90d61da62549208fe117b4384a469b" candidate_term: 22 candidate_status { last_received { term: 21 index: 3331 } } ignore_live_leader: false dest_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" is_pre_election: true
I20260812 06:37:05.359386  7322 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8d90d61da62549208fe117b4384a469b in term 21.
I20260812 06:37:05.363051 10694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "8d90d61da62549208fe117b4384a469b" candidate_term: 22 candidate_status { last_received { term: 21 index: 3331 } } ignore_live_leader: false dest_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" is_pre_election: true
W20260812 06:37:05.363446  6150 leader_election.cc:343] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b [CANDIDATE]: Term 22 pre-election: Tablet error from VoteRequest() call to peer 7962f587ccd842ad8a3f6ed9a5d7707e (127.4.177.3:40897): Illegal state: must be running to vote when last-logged opid is not known
I20260812 06:37:05.738801 10594 tablet_bootstrap.cc:492] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e: Bootstrap replayed 1/1 log segments. Stats: ops{read=3055 overwritten=27 applied=3020 ignored=0} inserts{seen=30050 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 8 replicates
I20260812 06:37:05.739478 10594 tablet_bootstrap.cc:492] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e: Bootstrap complete.
I20260812 06:37:05.740733 10594 ts_tablet_manager.cc:1403] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e: Time spent bootstrapping tablet: real 2.910s	user 2.124s	sys 0.163s
I20260812 06:37:05.742334 10594 raft_consensus.cc:359] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [term 20 FOLLOWER]: Replica starting. Triggering 8 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df4308c852240859ada53d216ab69ae" member_type: VOTER last_known_addr { host: "127.4.177.6" port: 34269 } } peers { permanent_uuid: "42895a9bcba64aa484f6509605035d02" member_type: VOTER last_known_addr { host: "127.4.177.4" port: 38181 } } peers { permanent_uuid: "222b20de5a1145c6b8a11df8374e8790" member_type: VOTER last_known_addr { host: "127.4.177.5" port: 43521 } } peers { permanent_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 40897 } } peers { permanent_uuid: "8d90d61da62549208fe117b4384a469b" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 41539 } } peers { permanent_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" member_type: VOTER last_known_addr { host: "127.4.177.7" port: 46299 } } peers { permanent_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 41057 } }
I20260812 06:37:05.744294 10594 raft_consensus.cc:740] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [term 20 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7962f587ccd842ad8a3f6ed9a5d7707e, State: Initialized, Role: FOLLOWER
I20260812 06:37:05.744719 10594 consensus_queue.cc:260] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3020, Last appended: 20.3028, Last appended by leader: 3028, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df4308c852240859ada53d216ab69ae" member_type: VOTER last_known_addr { host: "127.4.177.6" port: 34269 } } peers { permanent_uuid: "42895a9bcba64aa484f6509605035d02" member_type: VOTER last_known_addr { host: "127.4.177.4" port: 38181 } } peers { permanent_uuid: "222b20de5a1145c6b8a11df8374e8790" member_type: VOTER last_known_addr { host: "127.4.177.5" port: 43521 } } peers { permanent_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 40897 } } peers { permanent_uuid: "8d90d61da62549208fe117b4384a469b" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 41539 } } peers { permanent_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" member_type: VOTER last_known_addr { host: "127.4.177.7" port: 46299 } } peers { permanent_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 41057 } }
I20260812 06:37:05.745440 10594 ts_tablet_manager.cc:1434] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e: Time spent starting tablet: real 0.005s	user 0.006s	sys 0.000s
W20260812 06:37:06.835811  7211 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.4.177.4:38181 timed out after 2.059s (SENT)
I20260812 06:37:06.876536 10811 raft_consensus.cc:493] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 06:37:06.876729 10811 raft_consensus.cc:515] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df4308c852240859ada53d216ab69ae" member_type: VOTER last_known_addr { host: "127.4.177.6" port: 34269 } } peers { permanent_uuid: "42895a9bcba64aa484f6509605035d02" member_type: VOTER last_known_addr { host: "127.4.177.4" port: 38181 } } peers { permanent_uuid: "222b20de5a1145c6b8a11df8374e8790" member_type: VOTER last_known_addr { host: "127.4.177.5" port: 43521 } } peers { permanent_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 40897 } } peers { permanent_uuid: "8d90d61da62549208fe117b4384a469b" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 41539 } } peers { permanent_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" member_type: VOTER last_known_addr { host: "127.4.177.7" port: 46299 } } peers { permanent_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 41057 } }
I20260812 06:37:06.878031 10811 leader_election.cc:290] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 2df4308c852240859ada53d216ab69ae (127.4.177.6:34269), 42895a9bcba64aa484f6509605035d02 (127.4.177.4:38181), 222b20de5a1145c6b8a11df8374e8790 (127.4.177.5:43521), 8d90d61da62549208fe117b4384a469b (127.4.177.2:41539), b037d17b21f8422fa5e41b70eb4dcd08 (127.4.177.7:46299), ec6c8d818e744cdbb6b0d15b321010e0 (127.4.177.1:41057)
I20260812 06:37:06.883203  9797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" candidate_term: 21 candidate_status { last_received { term: 20 index: 3028 } } ignore_live_leader: false dest_uuid: "2df4308c852240859ada53d216ab69ae" is_pre_election: true
I20260812 06:37:06.883417  9797 raft_consensus.cc:2393] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7962f587ccd842ad8a3f6ed9a5d7707e in current term 21: Already voted for candidate 42895a9bcba64aa484f6509605035d02 in this term.
I20260812 06:37:06.884445  6236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" candidate_term: 21 candidate_status { last_received { term: 20 index: 3028 } } ignore_live_leader: false dest_uuid: "8d90d61da62549208fe117b4384a469b" is_pre_election: true
I20260812 06:37:06.884668  6236 raft_consensus.cc:2393] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7962f587ccd842ad8a3f6ed9a5d7707e in current term 21: Already voted for candidate 42895a9bcba64aa484f6509605035d02 in this term.
I20260812 06:37:06.884766  7322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" candidate_term: 21 candidate_status { last_received { term: 20 index: 3028 } } ignore_live_leader: false dest_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" is_pre_election: true
I20260812 06:37:06.884927  7322 raft_consensus.cc:2393] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7962f587ccd842ad8a3f6ed9a5d7707e in current term 21: Already voted for candidate 42895a9bcba64aa484f6509605035d02 in this term.
I20260812 06:37:06.885288  6048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" candidate_term: 21 candidate_status { last_received { term: 20 index: 3028 } } ignore_live_leader: false dest_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" is_pre_election: true
I20260812 06:37:06.885619  6048 raft_consensus.cc:2393] T 7065bb403e3047dd9a0853c49318ef1f P ec6c8d818e744cdbb6b0d15b321010e0 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7962f587ccd842ad8a3f6ed9a5d7707e in current term 21: Already voted for candidate 42895a9bcba64aa484f6509605035d02 in this term.
I20260812 06:37:06.886071 10588 leader_election.cc:304] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 5 responses out of 7 voters: 1 yes votes; 4 no votes. yes voters: 7962f587ccd842ad8a3f6ed9a5d7707e; no voters: 2df4308c852240859ada53d216ab69ae, 8d90d61da62549208fe117b4384a469b, b037d17b21f8422fa5e41b70eb4dcd08, ec6c8d818e744cdbb6b0d15b321010e0
I20260812 06:37:06.886420 10811 raft_consensus.cc:3060] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [term 20 FOLLOWER]: Advancing to term 21
I20260812 06:37:06.886763  9425 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" candidate_term: 21 candidate_status { last_received { term: 20 index: 3028 } } ignore_live_leader: false dest_uuid: "222b20de5a1145c6b8a11df8374e8790" is_pre_election: true
I20260812 06:37:06.886933  9425 raft_consensus.cc:2393] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7962f587ccd842ad8a3f6ed9a5d7707e in current term 21: Already voted for candidate 42895a9bcba64aa484f6509605035d02 in this term.
I20260812 06:37:06.888366 10811 raft_consensus.cc:2749] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260812 06:37:06.902288  9714 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.4.177.4:38181 timed out after 1.987s (SENT)
W20260812 06:37:07.042434  9315 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.4.177.4:38181 timed out after 2.124s (SENT)
I20260812 06:37:07.369239 10841 raft_consensus.cc:493] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 42895a9bcba64aa484f6509605035d02)
I20260812 06:37:07.369428 10841 raft_consensus.cc:515] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df4308c852240859ada53d216ab69ae" member_type: VOTER last_known_addr { host: "127.4.177.6" port: 34269 } } peers { permanent_uuid: "42895a9bcba64aa484f6509605035d02" member_type: VOTER last_known_addr { host: "127.4.177.4" port: 38181 } } peers { permanent_uuid: "222b20de5a1145c6b8a11df8374e8790" member_type: VOTER last_known_addr { host: "127.4.177.5" port: 43521 } } peers { permanent_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 40897 } } peers { permanent_uuid: "8d90d61da62549208fe117b4384a469b" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 41539 } } peers { permanent_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" member_type: VOTER last_known_addr { host: "127.4.177.7" port: 46299 } } peers { permanent_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 41057 } }
I20260812 06:37:07.370062 10841 leader_election.cc:290] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 2df4308c852240859ada53d216ab69ae (127.4.177.6:34269), 42895a9bcba64aa484f6509605035d02 (127.4.177.4:38181), 222b20de5a1145c6b8a11df8374e8790 (127.4.177.5:43521), 7962f587ccd842ad8a3f6ed9a5d7707e (127.4.177.3:40897), 8d90d61da62549208fe117b4384a469b (127.4.177.2:41539), ec6c8d818e744cdbb6b0d15b321010e0 (127.4.177.1:41057)
I20260812 06:37:07.370584  9797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" candidate_term: 22 candidate_status { last_received { term: 21 index: 3295 } } ignore_live_leader: false dest_uuid: "2df4308c852240859ada53d216ab69ae" is_pre_election: true
I20260812 06:37:07.370683  9425 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" candidate_term: 22 candidate_status { last_received { term: 21 index: 3295 } } ignore_live_leader: false dest_uuid: "222b20de5a1145c6b8a11df8374e8790" is_pre_election: true
I20260812 06:37:07.370767  9797 raft_consensus.cc:2410] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b037d17b21f8422fa5e41b70eb4dcd08 for term 22 because replica has last-logged OpId of term: 21 index: 3338, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 3295.
I20260812 06:37:07.370831  9425 raft_consensus.cc:2410] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b037d17b21f8422fa5e41b70eb4dcd08 for term 22 because replica has last-logged OpId of term: 21 index: 3338, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 3295.
I20260812 06:37:07.370774 10694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" candidate_term: 22 candidate_status { last_received { term: 21 index: 3295 } } ignore_live_leader: false dest_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" is_pre_election: true
I20260812 06:37:07.370976 10694 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b037d17b21f8422fa5e41b70eb4dcd08 in term 21.
I20260812 06:37:07.371029  6236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" candidate_term: 22 candidate_status { last_received { term: 21 index: 3295 } } ignore_live_leader: false dest_uuid: "8d90d61da62549208fe117b4384a469b" is_pre_election: true
I20260812 06:37:07.371093  6048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" candidate_term: 22 candidate_status { last_received { term: 21 index: 3295 } } ignore_live_leader: false dest_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" is_pre_election: true
I20260812 06:37:07.371184  6236 raft_consensus.cc:2410] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b037d17b21f8422fa5e41b70eb4dcd08 for term 22 because replica has last-logged OpId of term: 21 index: 3331, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 3295.
I20260812 06:37:07.371217  6048 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P ec6c8d818e744cdbb6b0d15b321010e0 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b037d17b21f8422fa5e41b70eb4dcd08 in term 21.
W20260812 06:37:07.414374  5945 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.4.177.4:38181 timed out after 2.061s (SENT)
W20260812 06:37:07.467868  6150 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.4.177.4:38181 timed out after 2.109s (SENT)
W20260812 06:37:08.766058 10588 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: connection negotiation to 127.4.177.4:38181 for RPC RequestConsensusVote timed out after 1.886s (ON_OUTBOUND_QUEUE)
W20260812 06:37:08.890902  9714 leader_election.cc:336] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 42895a9bcba64aa484f6509605035d02 (127.4.177.4:38181): Timed out: RequestConsensusVote RPC to 127.4.177.4:38181 timed out after 1.987s (SENT)
I20260812 06:37:08.891019  9714 leader_election.cc:304] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 7 responses out of 7 voters: 3 yes votes; 4 no votes. yes voters: 222b20de5a1145c6b8a11df8374e8790, 2df4308c852240859ada53d216ab69ae, b037d17b21f8422fa5e41b70eb4dcd08; no voters: 42895a9bcba64aa484f6509605035d02, 7962f587ccd842ad8a3f6ed9a5d7707e, 8d90d61da62549208fe117b4384a469b, ec6c8d818e744cdbb6b0d15b321010e0
I20260812 06:37:08.891428 10846 raft_consensus.cc:2749] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260812 06:37:08.896564  7211 leader_election.cc:336] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 42895a9bcba64aa484f6509605035d02 (127.4.177.4:38181): Timed out: RequestConsensusVote RPC to 127.4.177.4:38181 timed out after 2.059s (SENT)
I20260812 06:37:09.061991 10847 raft_consensus.cc:493] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 06:37:09.062140 10847 raft_consensus.cc:515] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df4308c852240859ada53d216ab69ae" member_type: VOTER last_known_addr { host: "127.4.177.6" port: 34269 } } peers { permanent_uuid: "42895a9bcba64aa484f6509605035d02" member_type: VOTER last_known_addr { host: "127.4.177.4" port: 38181 } } peers { permanent_uuid: "222b20de5a1145c6b8a11df8374e8790" member_type: VOTER last_known_addr { host: "127.4.177.5" port: 43521 } } peers { permanent_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 40897 } } peers { permanent_uuid: "8d90d61da62549208fe117b4384a469b" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 41539 } } peers { permanent_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" member_type: VOTER last_known_addr { host: "127.4.177.7" port: 46299 } } peers { permanent_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 41057 } }
I20260812 06:37:09.062994 10847 leader_election.cc:290] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 2df4308c852240859ada53d216ab69ae (127.4.177.6:34269), 42895a9bcba64aa484f6509605035d02 (127.4.177.4:38181), 222b20de5a1145c6b8a11df8374e8790 (127.4.177.5:43521), 8d90d61da62549208fe117b4384a469b (127.4.177.2:41539), b037d17b21f8422fa5e41b70eb4dcd08 (127.4.177.7:46299), ec6c8d818e744cdbb6b0d15b321010e0 (127.4.177.1:41057)
I20260812 06:37:09.063529  6048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" candidate_term: 22 candidate_status { last_received { term: 20 index: 3028 } } ignore_live_leader: false dest_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" is_pre_election: true
I20260812 06:37:09.063711  6048 raft_consensus.cc:2410] T 7065bb403e3047dd9a0853c49318ef1f P ec6c8d818e744cdbb6b0d15b321010e0 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7962f587ccd842ad8a3f6ed9a5d7707e for term 22 because replica has last-logged OpId of term: 21 index: 3272, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 3028.
I20260812 06:37:09.063652  9797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" candidate_term: 22 candidate_status { last_received { term: 20 index: 3028 } } ignore_live_leader: false dest_uuid: "2df4308c852240859ada53d216ab69ae" is_pre_election: true
I20260812 06:37:09.063836  9797 raft_consensus.cc:2410] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7962f587ccd842ad8a3f6ed9a5d7707e for term 22 because replica has last-logged OpId of term: 21 index: 3338, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 3028.
I20260812 06:37:09.063975  9425 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" candidate_term: 22 candidate_status { last_received { term: 20 index: 3028 } } ignore_live_leader: false dest_uuid: "222b20de5a1145c6b8a11df8374e8790" is_pre_election: true
I20260812 06:37:09.064123  9425 raft_consensus.cc:2410] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7962f587ccd842ad8a3f6ed9a5d7707e for term 22 because replica has last-logged OpId of term: 21 index: 3338, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 3028.
I20260812 06:37:09.064355  7322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" candidate_term: 22 candidate_status { last_received { term: 20 index: 3028 } } ignore_live_leader: false dest_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" is_pre_election: true
I20260812 06:37:09.064412  6236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" candidate_term: 22 candidate_status { last_received { term: 20 index: 3028 } } ignore_live_leader: false dest_uuid: "8d90d61da62549208fe117b4384a469b" is_pre_election: true
I20260812 06:37:09.064502  7322 raft_consensus.cc:2410] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7962f587ccd842ad8a3f6ed9a5d7707e for term 22 because replica has last-logged OpId of term: 21 index: 3295, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 3028.
I20260812 06:37:09.064574  6236 raft_consensus.cc:2410] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7962f587ccd842ad8a3f6ed9a5d7707e for term 22 because replica has last-logged OpId of term: 21 index: 3331, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 3028.
I20260812 06:37:09.065183 10588 leader_election.cc:304] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 5 responses out of 7 voters: 1 yes votes; 4 no votes. yes voters: 7962f587ccd842ad8a3f6ed9a5d7707e; no voters: 222b20de5a1145c6b8a11df8374e8790, 2df4308c852240859ada53d216ab69ae, 8d90d61da62549208fe117b4384a469b, b037d17b21f8422fa5e41b70eb4dcd08
I20260812 06:37:09.065404 10847 raft_consensus.cc:2749] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260812 06:37:09.168129  9315 leader_election.cc:336] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 42895a9bcba64aa484f6509605035d02 (127.4.177.4:38181): Timed out: RequestConsensusVote RPC to 127.4.177.4:38181 timed out after 2.124s (SENT)
I20260812 06:37:09.168246  9315 leader_election.cc:304] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 7 responses out of 7 voters: 3 yes votes; 4 no votes. yes voters: 222b20de5a1145c6b8a11df8374e8790, 2df4308c852240859ada53d216ab69ae, b037d17b21f8422fa5e41b70eb4dcd08; no voters: 42895a9bcba64aa484f6509605035d02, 7962f587ccd842ad8a3f6ed9a5d7707e, 8d90d61da62549208fe117b4384a469b, ec6c8d818e744cdbb6b0d15b321010e0
I20260812 06:37:09.168615 10849 raft_consensus.cc:2749] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260812 06:37:09.456167 10852 raft_consensus.cc:493] T 7065bb403e3047dd9a0853c49318ef1f P ec6c8d818e744cdbb6b0d15b321010e0 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 42895a9bcba64aa484f6509605035d02)
I20260812 06:37:09.456331 10852 raft_consensus.cc:515] T 7065bb403e3047dd9a0853c49318ef1f P ec6c8d818e744cdbb6b0d15b321010e0 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df4308c852240859ada53d216ab69ae" member_type: VOTER last_known_addr { host: "127.4.177.6" port: 34269 } } peers { permanent_uuid: "42895a9bcba64aa484f6509605035d02" member_type: VOTER last_known_addr { host: "127.4.177.4" port: 38181 } } peers { permanent_uuid: "222b20de5a1145c6b8a11df8374e8790" member_type: VOTER last_known_addr { host: "127.4.177.5" port: 43521 } } peers { permanent_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 40897 } } peers { permanent_uuid: "8d90d61da62549208fe117b4384a469b" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 41539 } } peers { permanent_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" member_type: VOTER last_known_addr { host: "127.4.177.7" port: 46299 } } peers { permanent_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 41057 } }
I20260812 06:37:09.456980 10852 leader_election.cc:290] T 7065bb403e3047dd9a0853c49318ef1f P ec6c8d818e744cdbb6b0d15b321010e0 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 2df4308c852240859ada53d216ab69ae (127.4.177.6:34269), 42895a9bcba64aa484f6509605035d02 (127.4.177.4:38181), 222b20de5a1145c6b8a11df8374e8790 (127.4.177.5:43521), 7962f587ccd842ad8a3f6ed9a5d7707e (127.4.177.3:40897), 8d90d61da62549208fe117b4384a469b (127.4.177.2:41539), b037d17b21f8422fa5e41b70eb4dcd08 (127.4.177.7:46299)
I20260812 06:37:09.457404  9797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" candidate_term: 22 candidate_status { last_received { term: 21 index: 3272 } } ignore_live_leader: false dest_uuid: "2df4308c852240859ada53d216ab69ae" is_pre_election: true
I20260812 06:37:09.457454  7322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" candidate_term: 22 candidate_status { last_received { term: 21 index: 3272 } } ignore_live_leader: false dest_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" is_pre_election: true
I20260812 06:37:09.457494  9425 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" candidate_term: 22 candidate_status { last_received { term: 21 index: 3272 } } ignore_live_leader: false dest_uuid: "222b20de5a1145c6b8a11df8374e8790" is_pre_election: true
I20260812 06:37:09.457615  9797 raft_consensus.cc:2410] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ec6c8d818e744cdbb6b0d15b321010e0 for term 22 because replica has last-logged OpId of term: 21 index: 3338, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 3272.
I20260812 06:37:09.457628  7322 raft_consensus.cc:2410] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ec6c8d818e744cdbb6b0d15b321010e0 for term 22 because replica has last-logged OpId of term: 21 index: 3295, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 3272.
I20260812 06:37:09.457653  9425 raft_consensus.cc:2410] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ec6c8d818e744cdbb6b0d15b321010e0 for term 22 because replica has last-logged OpId of term: 21 index: 3338, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 3272.
I20260812 06:37:09.457857 10694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" candidate_term: 22 candidate_status { last_received { term: 21 index: 3272 } } ignore_live_leader: false dest_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" is_pre_election: true
I20260812 06:37:09.458024 10694 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec6c8d818e744cdbb6b0d15b321010e0 in term 21.
I20260812 06:37:09.458033  6236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" candidate_term: 22 candidate_status { last_received { term: 21 index: 3272 } } ignore_live_leader: false dest_uuid: "8d90d61da62549208fe117b4384a469b" is_pre_election: true
I20260812 06:37:09.458228  6236 raft_consensus.cc:2410] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ec6c8d818e744cdbb6b0d15b321010e0 for term 22 because replica has last-logged OpId of term: 21 index: 3331, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 3272.
I20260812 06:37:09.458555  5945 leader_election.cc:304] T 7065bb403e3047dd9a0853c49318ef1f P ec6c8d818e744cdbb6b0d15b321010e0 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 6 responses out of 7 voters: 2 yes votes; 4 no votes. yes voters: 7962f587ccd842ad8a3f6ed9a5d7707e, ec6c8d818e744cdbb6b0d15b321010e0; no voters: 222b20de5a1145c6b8a11df8374e8790, 2df4308c852240859ada53d216ab69ae, 8d90d61da62549208fe117b4384a469b, b037d17b21f8422fa5e41b70eb4dcd08
I20260812 06:37:09.458794 10852 raft_consensus.cc:2749] T 7065bb403e3047dd9a0853c49318ef1f P ec6c8d818e744cdbb6b0d15b321010e0 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260812 06:37:09.476931  5945 leader_election.cc:336] T 7065bb403e3047dd9a0853c49318ef1f P ec6c8d818e744cdbb6b0d15b321010e0 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 42895a9bcba64aa484f6509605035d02 (127.4.177.4:38181): Timed out: RequestConsensusVote RPC to 127.4.177.4:38181 timed out after 2.061s (SENT)
W20260812 06:37:09.578910  6150 leader_election.cc:336] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 42895a9bcba64aa484f6509605035d02 (127.4.177.4:38181): Timed out: RequestConsensusVote RPC to 127.4.177.4:38181 timed out after 2.109s (SENT)
I20260812 06:37:09.579033  6150 leader_election.cc:304] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 7 responses out of 7 voters: 3 yes votes; 4 no votes. yes voters: 8d90d61da62549208fe117b4384a469b, b037d17b21f8422fa5e41b70eb4dcd08, ec6c8d818e744cdbb6b0d15b321010e0; no voters: 222b20de5a1145c6b8a11df8374e8790, 2df4308c852240859ada53d216ab69ae, 42895a9bcba64aa484f6509605035d02, 7962f587ccd842ad8a3f6ed9a5d7707e
I20260812 06:37:09.579438 10853 raft_consensus.cc:2749] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260812 06:37:10.019178 10826 negotiation.cc:336] Failed RPC negotiation. Trace:
0812 06:37:06.878712 (+     0us) reactor.cc:730] Submitting negotiation task for client connection to 127.4.177.4:38181 (local address 127.4.177.3:41709)
0812 06:37:06.879404 (+   692us) negotiation.cc:107] Waiting for socket to connect
0812 06:37:06.879419 (+    15us) client_negotiation.cc:175] Beginning negotiation
0812 06:37:06.879498 (+    79us) client_negotiation.cc:262] Sending NEGOTIATE NegotiatePB request
0812 06:37:10.018625 (+3139127us) negotiation.cc:326] Negotiation complete: Timed out: Client connection negotiation failed: client connection to 127.4.177.4:38181: received 0 of 4 requested bytes
Metrics: {"client-negotiator.queue_time_us":589,"thread_start_us":78,"threads_started":1}
W20260812 06:37:10.019603 10588 leader_election.cc:336] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 42895a9bcba64aa484f6509605035d02 (127.4.177.4:38181): Timed out: Client connection negotiation failed: client connection to 127.4.177.4:38181: received 0 of 4 requested bytes
I20260812 06:37:10.966732 10855 raft_consensus.cc:493] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 42895a9bcba64aa484f6509605035d02)
I20260812 06:37:10.966888 10855 raft_consensus.cc:515] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df4308c852240859ada53d216ab69ae" member_type: VOTER last_known_addr { host: "127.4.177.6" port: 34269 } } peers { permanent_uuid: "42895a9bcba64aa484f6509605035d02" member_type: VOTER last_known_addr { host: "127.4.177.4" port: 38181 } } peers { permanent_uuid: "222b20de5a1145c6b8a11df8374e8790" member_type: VOTER last_known_addr { host: "127.4.177.5" port: 43521 } } peers { permanent_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 40897 } } peers { permanent_uuid: "8d90d61da62549208fe117b4384a469b" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 41539 } } peers { permanent_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" member_type: VOTER last_known_addr { host: "127.4.177.7" port: 46299 } } peers { permanent_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 41057 } }
I20260812 06:37:10.967685 10855 leader_election.cc:290] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 42895a9bcba64aa484f6509605035d02 (127.4.177.4:38181), 222b20de5a1145c6b8a11df8374e8790 (127.4.177.5:43521), 7962f587ccd842ad8a3f6ed9a5d7707e (127.4.177.3:40897), 8d90d61da62549208fe117b4384a469b (127.4.177.2:41539), b037d17b21f8422fa5e41b70eb4dcd08 (127.4.177.7:46299), ec6c8d818e744cdbb6b0d15b321010e0 (127.4.177.1:41057)
I20260812 06:37:10.968184  7322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "2df4308c852240859ada53d216ab69ae" candidate_term: 22 candidate_status { last_received { term: 21 index: 3338 } } ignore_live_leader: false dest_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" is_pre_election: true
I20260812 06:37:10.968261 10694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "2df4308c852240859ada53d216ab69ae" candidate_term: 22 candidate_status { last_received { term: 21 index: 3338 } } ignore_live_leader: false dest_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" is_pre_election: true
I20260812 06:37:10.968261  9425 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "2df4308c852240859ada53d216ab69ae" candidate_term: 22 candidate_status { last_received { term: 21 index: 3338 } } ignore_live_leader: false dest_uuid: "222b20de5a1145c6b8a11df8374e8790" is_pre_election: true
I20260812 06:37:10.968343  7322 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2df4308c852240859ada53d216ab69ae in term 21.
I20260812 06:37:10.968406  9425 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2df4308c852240859ada53d216ab69ae in term 21.
I20260812 06:37:10.968412 10694 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2df4308c852240859ada53d216ab69ae in term 21.
I20260812 06:37:10.968362  6236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "2df4308c852240859ada53d216ab69ae" candidate_term: 22 candidate_status { last_received { term: 21 index: 3338 } } ignore_live_leader: false dest_uuid: "8d90d61da62549208fe117b4384a469b" is_pre_election: true
I20260812 06:37:10.968523  6236 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2df4308c852240859ada53d216ab69ae in term 21.
I20260812 06:37:10.968901  9714 leader_election.cc:304] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 7 voters: 4 yes votes; 0 no votes. yes voters: 222b20de5a1145c6b8a11df8374e8790, 2df4308c852240859ada53d216ab69ae, 8d90d61da62549208fe117b4384a469b, b037d17b21f8422fa5e41b70eb4dcd08; no voters: 
I20260812 06:37:10.968926  6048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "2df4308c852240859ada53d216ab69ae" candidate_term: 22 candidate_status { last_received { term: 21 index: 3338 } } ignore_live_leader: false dest_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" is_pre_election: true
I20260812 06:37:10.969091  6048 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P ec6c8d818e744cdbb6b0d15b321010e0 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2df4308c852240859ada53d216ab69ae in term 21.
I20260812 06:37:10.969135 10855 raft_consensus.cc:2804] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260812 06:37:10.969213 10855 raft_consensus.cc:493] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [term 21 FOLLOWER]: Starting leader election (detected failure of leader 42895a9bcba64aa484f6509605035d02)
I20260812 06:37:10.969278 10855 raft_consensus.cc:3060] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [term 21 FOLLOWER]: Advancing to term 22
I20260812 06:37:10.970189 10855 raft_consensus.cc:515] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df4308c852240859ada53d216ab69ae" member_type: VOTER last_known_addr { host: "127.4.177.6" port: 34269 } } peers { permanent_uuid: "42895a9bcba64aa484f6509605035d02" member_type: VOTER last_known_addr { host: "127.4.177.4" port: 38181 } } peers { permanent_uuid: "222b20de5a1145c6b8a11df8374e8790" member_type: VOTER last_known_addr { host: "127.4.177.5" port: 43521 } } peers { permanent_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 40897 } } peers { permanent_uuid: "8d90d61da62549208fe117b4384a469b" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 41539 } } peers { permanent_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" member_type: VOTER last_known_addr { host: "127.4.177.7" port: 46299 } } peers { permanent_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 41057 } }
I20260812 06:37:10.970736 10855 leader_election.cc:290] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [CANDIDATE]: Term 22 election: Requested vote from peers 42895a9bcba64aa484f6509605035d02 (127.4.177.4:38181), 222b20de5a1145c6b8a11df8374e8790 (127.4.177.5:43521), 7962f587ccd842ad8a3f6ed9a5d7707e (127.4.177.3:40897), 8d90d61da62549208fe117b4384a469b (127.4.177.2:41539), b037d17b21f8422fa5e41b70eb4dcd08 (127.4.177.7:46299), ec6c8d818e744cdbb6b0d15b321010e0 (127.4.177.1:41057)
I20260812 06:37:10.971064  7322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "2df4308c852240859ada53d216ab69ae" candidate_term: 22 candidate_status { last_received { term: 21 index: 3338 } } ignore_live_leader: false dest_uuid: "b037d17b21f8422fa5e41b70eb4dcd08"
I20260812 06:37:10.971133  9425 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "2df4308c852240859ada53d216ab69ae" candidate_term: 22 candidate_status { last_received { term: 21 index: 3338 } } ignore_live_leader: false dest_uuid: "222b20de5a1145c6b8a11df8374e8790"
I20260812 06:37:10.971184  7322 raft_consensus.cc:3060] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08 [term 21 FOLLOWER]: Advancing to term 22
I20260812 06:37:10.971256  9425 raft_consensus.cc:3060] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [term 21 FOLLOWER]: Advancing to term 22
I20260812 06:37:10.971272  6236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "2df4308c852240859ada53d216ab69ae" candidate_term: 22 candidate_status { last_received { term: 21 index: 3338 } } ignore_live_leader: false dest_uuid: "8d90d61da62549208fe117b4384a469b"
I20260812 06:37:10.971302 10694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "2df4308c852240859ada53d216ab69ae" candidate_term: 22 candidate_status { last_received { term: 21 index: 3338 } } ignore_live_leader: false dest_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e"
I20260812 06:37:10.971369  6236 raft_consensus.cc:3060] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b [term 21 FOLLOWER]: Advancing to term 22
I20260812 06:37:10.971418 10694 raft_consensus.cc:3060] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [term 21 FOLLOWER]: Advancing to term 22
I20260812 06:37:10.972247  7322 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2df4308c852240859ada53d216ab69ae in term 22.
I20260812 06:37:10.972254  9425 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2df4308c852240859ada53d216ab69ae in term 22.
I20260812 06:37:10.972442  6236 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2df4308c852240859ada53d216ab69ae in term 22.
I20260812 06:37:10.972558 10694 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2df4308c852240859ada53d216ab69ae in term 22.
I20260812 06:37:10.972666  9714 leader_election.cc:304] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 7 voters: 4 yes votes; 0 no votes. yes voters: 222b20de5a1145c6b8a11df8374e8790, 2df4308c852240859ada53d216ab69ae, 8d90d61da62549208fe117b4384a469b, b037d17b21f8422fa5e41b70eb4dcd08; no voters: 
I20260812 06:37:10.972765  6048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "2df4308c852240859ada53d216ab69ae" candidate_term: 22 candidate_status { last_received { term: 21 index: 3338 } } ignore_live_leader: false dest_uuid: "ec6c8d818e744cdbb6b0d15b321010e0"
I20260812 06:37:10.972882 10855 raft_consensus.cc:2804] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [term 22 FOLLOWER]: Leader election won for term 22
I20260812 06:37:10.972895  6048 raft_consensus.cc:3060] T 7065bb403e3047dd9a0853c49318ef1f P ec6c8d818e744cdbb6b0d15b321010e0 [term 21 FOLLOWER]: Advancing to term 22
I20260812 06:37:10.973089 10855 raft_consensus.cc:697] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [term 22 LEADER]: Becoming Leader. State: Replica: 2df4308c852240859ada53d216ab69ae, State: Running, Role: LEADER
I20260812 06:37:10.973278 10855 consensus_queue.cc:237] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3330, Committed index: 3330, Last appended: 21.3338, Last appended by leader: 3338, Current term: 22, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df4308c852240859ada53d216ab69ae" member_type: VOTER last_known_addr { host: "127.4.177.6" port: 34269 } } peers { permanent_uuid: "42895a9bcba64aa484f6509605035d02" member_type: VOTER last_known_addr { host: "127.4.177.4" port: 38181 } } peers { permanent_uuid: "222b20de5a1145c6b8a11df8374e8790" member_type: VOTER last_known_addr { host: "127.4.177.5" port: 43521 } } peers { permanent_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 40897 } } peers { permanent_uuid: "8d90d61da62549208fe117b4384a469b" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 41539 } } peers { permanent_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" member_type: VOTER last_known_addr { host: "127.4.177.7" port: 46299 } } peers { permanent_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 41057 } }
I20260812 06:37:10.973871  6048 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P ec6c8d818e744cdbb6b0d15b321010e0 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2df4308c852240859ada53d216ab69ae in term 22.
I20260812 06:37:10.975764  5814 catalog_manager.cc:5719] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae reported cstate change: term changed from 21 to 22, leader changed from 42895a9bcba64aa484f6509605035d02 (127.4.177.4) to 2df4308c852240859ada53d216ab69ae (127.4.177.6). New cstate: current_term: 22 leader_uuid: "2df4308c852240859ada53d216ab69ae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df4308c852240859ada53d216ab69ae" member_type: VOTER last_known_addr { host: "127.4.177.6" port: 34269 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "42895a9bcba64aa484f6509605035d02" member_type: VOTER last_known_addr { host: "127.4.177.4" port: 38181 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "222b20de5a1145c6b8a11df8374e8790" member_type: VOTER last_known_addr { host: "127.4.177.5" port: 43521 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 40897 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8d90d61da62549208fe117b4384a469b" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 41539 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" member_type: VOTER last_known_addr { host: "127.4.177.7" port: 46299 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 41057 } health_report { overall_health: UNKNOWN } } }
W20260812 06:37:11.378660  7211 leader_election.cc:336] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 42895a9bcba64aa484f6509605035d02 (127.4.177.4:38181): Timed out: RequestConsensusVote RPC to 127.4.177.4:38181 timed out after 2.003s (SENT)
I20260812 06:37:11.378778  7211 leader_election.cc:304] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 7 responses out of 7 voters: 3 yes votes; 4 no votes. yes voters: 7962f587ccd842ad8a3f6ed9a5d7707e, b037d17b21f8422fa5e41b70eb4dcd08, ec6c8d818e744cdbb6b0d15b321010e0; no voters: 222b20de5a1145c6b8a11df8374e8790, 2df4308c852240859ada53d216ab69ae, 42895a9bcba64aa484f6509605035d02, 8d90d61da62549208fe117b4384a469b
I20260812 06:37:11.379199 10867 raft_consensus.cc:2749] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260812 06:37:11.436477  9425 raft_consensus.cc:1275] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [term 22 FOLLOWER]: Refusing update from remote peer 2df4308c852240859ada53d216ab69ae: Log matching property violated. Preceding OpId in replica: term: 21 index: 3338. Preceding OpId from leader: term: 22 index: 3339. (index mismatch)
I20260812 06:37:11.437186 10855 consensus_queue.cc:1048] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "222b20de5a1145c6b8a11df8374e8790" member_type: VOTER last_known_addr { host: "127.4.177.5" port: 43521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3339, Last known committed idx: 3330, Time since last communication: 0.000s
I20260812 06:37:11.446297  7322 raft_consensus.cc:1275] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08 [term 22 FOLLOWER]: Refusing update from remote peer 2df4308c852240859ada53d216ab69ae: Log matching property violated. Preceding OpId in replica: term: 21 index: 3295. Preceding OpId from leader: term: 22 index: 3339. (index mismatch)
I20260812 06:37:11.446863 10855 consensus_queue.cc:1048] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" member_type: VOTER last_known_addr { host: "127.4.177.7" port: 46299 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3339, Last known committed idx: 3292, Time since last communication: 0.000s
W20260812 06:37:11.460796  6595 connection.cc:256] RPC call timeout handler was delayed by -7.2931987850461155s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1s for a response
W20260812 06:37:11.461601  6597 connection.cc:256] RPC call timeout handler was delayed by -7.3910559471696615s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1s for a response
W20260812 06:37:11.461680  6597 connection.cc:256] RPC call timeout handler was delayed by -7.1665958741214126s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1s for a response
W20260812 06:37:11.461721  6597 connection.cc:256] RPC call timeout handler was delayed by -7.0164781759958714s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1s for a response
I20260812 06:37:11.462670  6668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "2df4308c852240859ada53d216ab69ae" candidate_term: 22 candidate_status { last_received { term: 21 index: 3338 } } ignore_live_leader: false dest_uuid: "42895a9bcba64aa484f6509605035d02" is_pre_election: true
I20260812 06:37:11.465070  6703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" candidate_term: 22 candidate_status { last_received { term: 21 index: 3272 } } ignore_live_leader: false dest_uuid: "42895a9bcba64aa484f6509605035d02" is_pre_election: true
I20260812 06:37:11.465327  6669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" candidate_term: 22 candidate_status { last_received { term: 21 index: 3295 } } ignore_live_leader: false dest_uuid: "42895a9bcba64aa484f6509605035d02" is_pre_election: true
I20260812 06:37:11.465541  6670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" candidate_term: 22 candidate_status { last_received { term: 21 index: 3295 } } ignore_live_leader: false dest_uuid: "42895a9bcba64aa484f6509605035d02" is_pre_election: true
I20260812 06:37:11.465869  6672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "2df4308c852240859ada53d216ab69ae" candidate_term: 22 candidate_status { last_received { term: 21 index: 3338 } } ignore_live_leader: false dest_uuid: "42895a9bcba64aa484f6509605035d02" is_pre_election: true
I20260812 06:37:11.466051  6673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" candidate_term: 22 candidate_status { last_received { term: 21 index: 3272 } } ignore_live_leader: false dest_uuid: "42895a9bcba64aa484f6509605035d02" is_pre_election: true
I20260812 06:37:11.466241  6680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "2df4308c852240859ada53d216ab69ae" candidate_term: 22 candidate_status { last_received { term: 21 index: 3338 } } ignore_live_leader: false dest_uuid: "42895a9bcba64aa484f6509605035d02"
I20260812 06:37:11.466427  6681 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "2df4308c852240859ada53d216ab69ae" candidate_term: 22 candidate_status { last_received { term: 21 index: 3338 } } ignore_live_leader: false dest_uuid: "42895a9bcba64aa484f6509605035d02" is_pre_election: true
I20260812 06:37:11.466600  6686 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" candidate_term: 22 candidate_status { last_received { term: 21 index: 3272 } } ignore_live_leader: false dest_uuid: "42895a9bcba64aa484f6509605035d02" is_pre_election: true
I20260812 06:37:11.467069  6691 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "8d90d61da62549208fe117b4384a469b" candidate_term: 22 candidate_status { last_received { term: 21 index: 3331 } } ignore_live_leader: false dest_uuid: "42895a9bcba64aa484f6509605035d02" is_pre_election: true
I20260812 06:37:11.467255  6692 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "222b20de5a1145c6b8a11df8374e8790" candidate_term: 22 candidate_status { last_received { term: 21 index: 3338 } } ignore_live_leader: false dest_uuid: "42895a9bcba64aa484f6509605035d02" is_pre_election: true
I20260812 06:37:11.467587  6694 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "222b20de5a1145c6b8a11df8374e8790" candidate_term: 22 candidate_status { last_received { term: 21 index: 3338 } } ignore_live_leader: false dest_uuid: "42895a9bcba64aa484f6509605035d02" is_pre_election: true
I20260812 06:37:11.467940  6699 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" candidate_term: 22 candidate_status { last_received { term: 21 index: 3295 } } ignore_live_leader: false dest_uuid: "42895a9bcba64aa484f6509605035d02" is_pre_election: true
I20260812 06:37:11.468127  6700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" candidate_term: 22 candidate_status { last_received { term: 21 index: 3295 } } ignore_live_leader: false dest_uuid: "42895a9bcba64aa484f6509605035d02" is_pre_election: true
I20260812 06:37:11.468302  6701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "8d90d61da62549208fe117b4384a469b" candidate_term: 22 candidate_status { last_received { term: 21 index: 3331 } } ignore_live_leader: false dest_uuid: "42895a9bcba64aa484f6509605035d02" is_pre_election: true
I20260812 06:37:11.468518 10529 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:42408 (ReqId={client: 3067f3aef2a44cc4bb7f2ccd6acf83db, seq_no=3314, attempt_no=0}) took 8080 ms. Trace:
I20260812 06:37:11.468595 10529 rpcz_store.cc:276] 0812 06:37:03.388132 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 06:37:03.388195 (+    63us) service_pool.cc:224] Handling call
0812 06:37:11.468494 (+8080299us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0812 06:37:03.388535 (+     0us) write_op.cc:183] PREPARE: starting on tablet 7065bb403e3047dd9a0853c49318ef1f
0812 06:37:03.388611 (+    76us) write_op.cc:432] Acquiring schema lock in shared mode
0812 06:37:03.388614 (+     3us) write_op.cc:435] Acquired schema lock
0812 06:37:03.388616 (+     2us) tablet.cc:662] Decoding operations
0812 06:37:03.388697 (+    81us) write_op.cc:620] Acquiring the partition lock for write op
0812 06:37:03.388703 (+     6us) write_op.cc:641] Partition lock acquired for write op
0812 06:37:03.388705 (+     2us) tablet.cc:685] Acquiring locks for 10 operations
0812 06:37:03.388768 (+    63us) tablet.cc:701] Row locks acquired
0812 06:37:03.388769 (+     1us) write_op.cc:260] PREPARE: finished
0812 06:37:03.388796 (+    27us) write_op.cc:270] Start()
0812 06:37:03.388814 (+    18us) write_op.cc:276] Timestamp: P: 1786516623388774 usec, L: 0
0812 06:37:03.388815 (+     1us) op_driver.cc:348] REPLICATION: starting
0812 06:37:03.388929 (+   114us) log.cc:844] Serialized 696 byte log entry
0812 06:37:11.467879 (+8078950us) op_driver.cc:464] REPLICATION: finished
0812 06:37:11.468029 (+   150us) write_op.cc:301] APPLY: starting
0812 06:37:11.468045 (+    16us) tablet.cc:1367] starting BulkCheckPresence
0812 06:37:11.468076 (+    31us) tablet.cc:1370] finished BulkCheckPresence
0812 06:37:11.468088 (+    12us) tablet.cc:1372] starting ApplyRowOperation cycle
0812 06:37:11.468267 (+   179us) tablet.cc:1383] finished ApplyRowOperation cycle
0812 06:37:11.468298 (+    31us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0812 06:37:11.468301 (+     3us) write_op.cc:312] APPLY: finished
0812 06:37:11.468369 (+    68us) log.cc:844] Serialized 77 byte log entry
0812 06:37:11.468411 (+    42us) write_op.cc:489] Releasing partition, row and schema locks
0812 06:37:11.468429 (+    18us) write_op.cc:454] Released schema lock
0812 06:37:11.468434 (+     5us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":98,"num_ops":10,"prepare.queue_time_us":44,"prepare.run_cpu_time_us":467,"prepare.run_wall_time_us":701,"raft.queue_time_us":31,"raft.run_cpu_time_us":4,"raft.run_wall_time_us":4,"replication_time_us":8079038}]]}
I20260812 06:37:11.469813  9425 raft_consensus.cc:1240] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [term 22 FOLLOWER]: Rejecting Update request from peer 42895a9bcba64aa484f6509605035d02 for earlier term 21. Current term is 22. Ops: []
I20260812 06:37:11.470649  6236 raft_consensus.cc:1240] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b [term 22 FOLLOWER]: Rejecting Update request from peer 42895a9bcba64aa484f6509605035d02 for earlier term 21. Current term is 22. Ops: [21.3332-21.3338]
I20260812 06:37:11.470191  9797 raft_consensus.cc:1240] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [term 22 LEADER]: Rejecting Update request from peer 42895a9bcba64aa484f6509605035d02 for earlier term 21. Current term is 22. Ops: []
I20260812 06:37:11.472754  6046 raft_consensus.cc:1240] T 7065bb403e3047dd9a0853c49318ef1f P ec6c8d818e744cdbb6b0d15b321010e0 [term 22 FOLLOWER]: Rejecting Update request from peer 42895a9bcba64aa484f6509605035d02 for earlier term 21. Current term is 22. Ops: [21.3273-21.3338]
I20260812 06:37:11.473168  7322 raft_consensus.cc:1217] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08 [term 22 FOLLOWER]: Deduplicated request from leader. Original: 21.3295->[21.3296-21.3338]   Dedup: 21.3338->[]
I20260812 06:37:11.473304  7322 raft_consensus.cc:1240] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08 [term 22 FOLLOWER]: Rejecting Update request from peer 42895a9bcba64aa484f6509605035d02 for earlier term 21. Current term is 22. Ops: [21.3296-21.3338]
I20260812 06:37:11.473711 10458 consensus_queue.cc:1059] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" member_type: VOTER last_known_addr { host: "127.4.177.7" port: 46299 }, Status: INVALID_TERM, Last received: 21.3295, Next index: 3296, Last known committed idx: 3330, Time since last communication: 0.000s
I20260812 06:37:11.474157 10531 consensus_queue.cc:1059] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 41057 }, Status: INVALID_TERM, Last received: 21.3272, Next index: 3273, Last known committed idx: 3271, Time since last communication: 0.000s
I20260812 06:37:11.474327 10535 consensus_queue.cc:1059] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2df4308c852240859ada53d216ab69ae" member_type: VOTER last_known_addr { host: "127.4.177.6" port: 34269 }, Status: INVALID_TERM, Last received: 21.3338, Next index: 3339, Last known committed idx: 3330, Time since last communication: 0.000s
I20260812 06:37:11.474424 10535 consensus_queue.cc:1243] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [LEADER]: Peer 2df4308c852240859ada53d216ab69ae log is divergent from this leader: its last log entry 22.3339 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3330
I20260812 06:37:11.474498 10535 consensus_queue.cc:1059] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8d90d61da62549208fe117b4384a469b" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 41539 }, Status: INVALID_TERM, Last received: 21.3331, Next index: 3332, Last known committed idx: 3323, Time since last communication: 0.000s
I20260812 06:37:11.474617 10531 consensus_queue.cc:1059] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "222b20de5a1145c6b8a11df8374e8790" member_type: VOTER last_known_addr { host: "127.4.177.5" port: 43521 }, Status: INVALID_TERM, Last received: 21.3338, Next index: 3339, Last known committed idx: 3330, Time since last communication: 0.000s
I20260812 06:37:11.474897 10458 raft_consensus.cc:3055] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [term 21 LEADER]: Stepping down as leader of term 21
I20260812 06:37:11.474956 10458 raft_consensus.cc:740] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 42895a9bcba64aa484f6509605035d02, State: Running, Role: LEADER
I20260812 06:37:11.475114 10458 consensus_queue.cc:260] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3331, Committed index: 3331, Last appended: 21.3339, Last appended by leader: 3339, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df4308c852240859ada53d216ab69ae" member_type: VOTER last_known_addr { host: "127.4.177.6" port: 34269 } } peers { permanent_uuid: "42895a9bcba64aa484f6509605035d02" member_type: VOTER last_known_addr { host: "127.4.177.4" port: 38181 } } peers { permanent_uuid: "222b20de5a1145c6b8a11df8374e8790" member_type: VOTER last_known_addr { host: "127.4.177.5" port: 43521 } } peers { permanent_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 40897 } } peers { permanent_uuid: "8d90d61da62549208fe117b4384a469b" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 41539 } } peers { permanent_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" member_type: VOTER last_known_addr { host: "127.4.177.7" port: 46299 } } peers { permanent_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 41057 } }
I20260812 06:37:11.475486 10458 raft_consensus.cc:3060] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [term 21 FOLLOWER]: Advancing to term 22
W20260812 06:37:11.478847 10882 negotiation.cc:336] Failed RPC negotiation. Trace:
0812 06:37:11.465061 (+     0us) reactor.cc:730] Submitting negotiation task for server connection from 127.4.177.3:41709 (local address 127.4.177.4:38181)
0812 06:37:11.477567 (+ 12506us) server_negotiation.cc:207] Beginning negotiation
0812 06:37:11.477571 (+     4us) server_negotiation.cc:400] Waiting for connection header
0812 06:37:11.477582 (+    11us) server_negotiation.cc:408] Connection header received
0812 06:37:11.477617 (+    35us) server_negotiation.cc:366] Received NEGOTIATE NegotiatePB request
0812 06:37:11.477621 (+     4us) server_negotiation.cc:462] Received NEGOTIATE request from client
0812 06:37:11.477654 (+    33us) server_negotiation.cc:378] Sending NEGOTIATE NegotiatePB response
0812 06:37:11.477790 (+   136us) negotiation.cc:326] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.177.3:41709: BlockingWrite error: write error: Broken pipe (error 32)
Metrics: {"server-negotiator.queue_time_us":12443,"thread_start_us":66,"threads_started":1}
I20260812 06:37:11.480712  6700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" candidate_term: 22 candidate_status { last_received { term: 20 index: 3028 } } ignore_live_leader: false dest_uuid: "42895a9bcba64aa484f6509605035d02" is_pre_election: true
I20260812 06:37:11.480854  6700 raft_consensus.cc:2410] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7962f587ccd842ad8a3f6ed9a5d7707e for term 22 because replica has last-logged OpId of term: 21 index: 3339, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 3028.
I20260812 06:37:11.501951  6700 raft_consensus.cc:1275] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [term 22 FOLLOWER]: Refusing update from remote peer 2df4308c852240859ada53d216ab69ae: Log matching property violated. Preceding OpId in replica: term: 21 index: 3339. Preceding OpId from leader: term: 22 index: 3339. (term mismatch)
I20260812 06:37:11.502067  6700 pending_rounds.cc:85] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02: Aborting all ops after (but not including) 3338
I20260812 06:37:11.502099  6700 pending_rounds.cc:107] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02: Aborting uncommitted WRITE_OP operation due to leader change: 21.3339
W20260812 06:37:11.502298  6700 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:42408: Aborted: Op aborted by new leader
I20260812 06:37:11.503542 10855 consensus_queue.cc:1048] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "42895a9bcba64aa484f6509605035d02" member_type: VOTER last_known_addr { host: "127.4.177.4" port: 38181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3339, Last known committed idx: 3331, Time since last communication: 0.000s
I20260812 06:37:11.506778 10529 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:42408 (ReqId={client: 3067f3aef2a44cc4bb7f2ccd6acf83db, seq_no=3315, attempt_no=0}) took 8103 ms. Trace:
I20260812 06:37:11.506871 10529 rpcz_store.cc:276] 0812 06:37:03.402854 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 06:37:03.402933 (+    79us) service_pool.cc:224] Handling call
0812 06:37:11.506766 (+8103833us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0812 06:37:03.403288 (+     0us) write_op.cc:183] PREPARE: starting on tablet 7065bb403e3047dd9a0853c49318ef1f
0812 06:37:03.403374 (+    86us) write_op.cc:432] Acquiring schema lock in shared mode
0812 06:37:03.403377 (+     3us) write_op.cc:435] Acquired schema lock
0812 06:37:03.403379 (+     2us) tablet.cc:662] Decoding operations
0812 06:37:03.403466 (+    87us) write_op.cc:620] Acquiring the partition lock for write op
0812 06:37:03.403472 (+     6us) write_op.cc:641] Partition lock acquired for write op
0812 06:37:03.403475 (+     3us) tablet.cc:685] Acquiring locks for 10 operations
0812 06:37:03.403548 (+    73us) tablet.cc:701] Row locks acquired
0812 06:37:03.403549 (+     1us) write_op.cc:260] PREPARE: finished
0812 06:37:03.403584 (+    35us) write_op.cc:270] Start()
0812 06:37:03.403604 (+    20us) write_op.cc:276] Timestamp: P: 1786516623403561 usec, L: 0
0812 06:37:03.403606 (+     2us) op_driver.cc:348] REPLICATION: starting
0812 06:37:03.403717 (+   111us) log.cc:844] Serialized 696 byte log entry
0812 06:37:11.506066 (+8102349us) op_driver.cc:464] REPLICATION: finished
0812 06:37:11.506199 (+   133us) write_op.cc:301] APPLY: starting
0812 06:37:11.506244 (+    45us) tablet.cc:1367] starting BulkCheckPresence
0812 06:37:11.506291 (+    47us) tablet.cc:1370] finished BulkCheckPresence
0812 06:37:11.506309 (+    18us) tablet.cc:1372] starting ApplyRowOperation cycle
0812 06:37:11.506558 (+   249us) tablet.cc:1383] finished ApplyRowOperation cycle
0812 06:37:11.506589 (+    31us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0812 06:37:11.506594 (+     5us) write_op.cc:312] APPLY: finished
0812 06:37:11.506647 (+    53us) log.cc:844] Serialized 77 byte log entry
0812 06:37:11.506682 (+    35us) write_op.cc:489] Releasing partition, row and schema locks
0812 06:37:11.506695 (+    13us) write_op.cc:454] Released schema lock
0812 06:37:11.506702 (+     7us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":82,"num_ops":10,"prepare.queue_time_us":47,"prepare.run_cpu_time_us":508,"prepare.run_wall_time_us":779,"raft.queue_time_us":33,"raft.run_cpu_time_us":4,"raft.run_wall_time_us":5,"replication_time_us":8102435,"spinlock_wait_cycles":3968}]]}
I20260812 06:37:11.507251 10528 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:42408 (ReqId={client: 3067f3aef2a44cc4bb7f2ccd6acf83db, seq_no=3317, attempt_no=0}) took 8104 ms. Trace:
I20260812 06:37:11.507309 10528 rpcz_store.cc:276] 0812 06:37:03.403009 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 06:37:03.403277 (+   268us) service_pool.cc:224] Handling call
0812 06:37:11.507237 (+8103960us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0812 06:37:03.404085 (+     0us) write_op.cc:183] PREPARE: starting on tablet 7065bb403e3047dd9a0853c49318ef1f
0812 06:37:03.404153 (+    68us) write_op.cc:432] Acquiring schema lock in shared mode
0812 06:37:03.404157 (+     4us) write_op.cc:435] Acquired schema lock
0812 06:37:03.404159 (+     2us) tablet.cc:662] Decoding operations
0812 06:37:03.404244 (+    85us) write_op.cc:620] Acquiring the partition lock for write op
0812 06:37:03.404250 (+     6us) write_op.cc:641] Partition lock acquired for write op
0812 06:37:03.404253 (+     3us) tablet.cc:685] Acquiring locks for 10 operations
0812 06:37:03.404329 (+    76us) tablet.cc:701] Row locks acquired
0812 06:37:03.404331 (+     2us) write_op.cc:260] PREPARE: finished
0812 06:37:03.404365 (+    34us) write_op.cc:270] Start()
0812 06:37:03.404388 (+    23us) write_op.cc:276] Timestamp: P: 1786516623404338 usec, L: 0
0812 06:37:03.404390 (+     2us) op_driver.cc:348] REPLICATION: starting
0812 06:37:03.404496 (+   106us) log.cc:844] Serialized 696 byte log entry
0812 06:37:11.506132 (+8101636us) op_driver.cc:464] REPLICATION: finished
0812 06:37:11.506773 (+   641us) write_op.cc:301] APPLY: starting
0812 06:37:11.506784 (+    11us) tablet.cc:1367] starting BulkCheckPresence
0812 06:37:11.506825 (+    41us) tablet.cc:1370] finished BulkCheckPresence
0812 06:37:11.506839 (+    14us) tablet.cc:1372] starting ApplyRowOperation cycle
0812 06:37:11.507049 (+   210us) tablet.cc:1383] finished ApplyRowOperation cycle
0812 06:37:11.507077 (+    28us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0812 06:37:11.507081 (+     4us) write_op.cc:312] APPLY: finished
0812 06:37:11.507135 (+    54us) log.cc:844] Serialized 77 byte log entry
0812 06:37:11.507169 (+    34us) write_op.cc:489] Releasing partition, row and schema locks
0812 06:37:11.507183 (+    14us) write_op.cc:454] Released schema lock
0812 06:37:11.507187 (+     4us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"spinlock_wait_cycles":21504,"child_traces":[["op",{"apply.queue_time_us":607,"num_ops":10,"prepare.queue_time_us":613,"prepare.run_cpu_time_us":492,"prepare.run_wall_time_us":1808,"raft.queue_time_us":25,"raft.run_cpu_time_us":17,"raft.run_wall_time_us":4,"replication_time_us":8101730}]]}
I20260812 06:37:11.507623 10529 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:42408 (ReqId={client: 3067f3aef2a44cc4bb7f2ccd6acf83db, seq_no=3321, attempt_no=0}) took 8104 ms. Trace:
I20260812 06:37:11.507673 10529 rpcz_store.cc:276] 0812 06:37:03.403220 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 06:37:03.403259 (+    39us) service_pool.cc:224] Handling call
0812 06:37:11.507614 (+8104355us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0812 06:37:03.405936 (+     0us) write_op.cc:183] PREPARE: starting on tablet 7065bb403e3047dd9a0853c49318ef1f
0812 06:37:03.406013 (+    77us) write_op.cc:432] Acquiring schema lock in shared mode
0812 06:37:03.406016 (+     3us) write_op.cc:435] Acquired schema lock
0812 06:37:03.406018 (+     2us) tablet.cc:662] Decoding operations
0812 06:37:03.406101 (+    83us) write_op.cc:620] Acquiring the partition lock for write op
0812 06:37:03.406106 (+     5us) write_op.cc:641] Partition lock acquired for write op
0812 06:37:03.406109 (+     3us) tablet.cc:685] Acquiring locks for 10 operations
0812 06:37:03.406169 (+    60us) tablet.cc:701] Row locks acquired
0812 06:37:03.406170 (+     1us) write_op.cc:260] PREPARE: finished
0812 06:37:03.406195 (+    25us) write_op.cc:270] Start()
0812 06:37:03.406247 (+    52us) write_op.cc:276] Timestamp: P: 1786516623406172 usec, L: 0
0812 06:37:03.406250 (+     3us) op_driver.cc:348] REPLICATION: starting
0812 06:37:03.406353 (+   103us) log.cc:844] Serialized 696 byte log entry
0812 06:37:11.506180 (+8099827us) op_driver.cc:464] REPLICATION: finished
0812 06:37:11.507216 (+  1036us) write_op.cc:301] APPLY: starting
0812 06:37:11.507224 (+     8us) tablet.cc:1367] starting BulkCheckPresence
0812 06:37:11.507249 (+    25us) tablet.cc:1370] finished BulkCheckPresence
0812 06:37:11.507260 (+    11us) tablet.cc:1372] starting ApplyRowOperation cycle
0812 06:37:11.507430 (+   170us) tablet.cc:1383] finished ApplyRowOperation cycle
0812 06:37:11.507466 (+    36us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0812 06:37:11.507471 (+     5us) write_op.cc:312] APPLY: finished
0812 06:37:11.507521 (+    50us) log.cc:844] Serialized 77 byte log entry
0812 06:37:11.507553 (+    32us) write_op.cc:489] Releasing partition, row and schema locks
0812 06:37:11.507566 (+    13us) write_op.cc:454] Released schema lock
0812 06:37:11.507570 (+     4us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":1010,"num_ops":10,"prepare.queue_time_us":2420,"prepare.run_cpu_time_us":514,"prepare.run_wall_time_us":959,"raft.queue_time_us":33,"raft.run_cpu_time_us":5,"raft.run_wall_time_us":5,"replication_time_us":8099917,"thread_start_us":97,"threads_started":1}]]}
I20260812 06:37:11.508028 10528 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:42408 (ReqId={client: 3067f3aef2a44cc4bb7f2ccd6acf83db, seq_no=3318, attempt_no=0}) took 8104 ms. Trace:
I20260812 06:37:11.508080 10528 rpcz_store.cc:276] 0812 06:37:03.403062 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 06:37:03.403509 (+   447us) service_pool.cc:224] Handling call
0812 06:37:11.508019 (+8104510us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0812 06:37:03.406913 (+     0us) write_op.cc:183] PREPARE: starting on tablet 7065bb403e3047dd9a0853c49318ef1f
0812 06:37:03.407245 (+   332us) write_op.cc:432] Acquiring schema lock in shared mode
0812 06:37:03.407251 (+     6us) write_op.cc:435] Acquired schema lock
0812 06:37:03.407254 (+     3us) tablet.cc:662] Decoding operations
0812 06:37:03.407352 (+    98us) write_op.cc:620] Acquiring the partition lock for write op
0812 06:37:03.407359 (+     7us) write_op.cc:641] Partition lock acquired for write op
0812 06:37:03.407362 (+     3us) tablet.cc:685] Acquiring locks for 10 operations
0812 06:37:03.407427 (+    65us) tablet.cc:701] Row locks acquired
0812 06:37:03.407429 (+     2us) write_op.cc:260] PREPARE: finished
0812 06:37:03.407459 (+    30us) write_op.cc:270] Start()
0812 06:37:03.407481 (+    22us) write_op.cc:276] Timestamp: P: 1786516623407434 usec, L: 0
0812 06:37:03.407484 (+     3us) op_driver.cc:348] REPLICATION: starting
0812 06:37:03.407589 (+   105us) log.cc:844] Serialized 696 byte log entry
0812 06:37:11.506331 (+8098742us) op_driver.cc:464] REPLICATION: finished
0812 06:37:11.507630 (+  1299us) write_op.cc:301] APPLY: starting
0812 06:37:11.507641 (+    11us) tablet.cc:1367] starting BulkCheckPresence
0812 06:37:11.507669 (+    28us) tablet.cc:1370] finished BulkCheckPresence
0812 06:37:11.507682 (+    13us) tablet.cc:1372] starting ApplyRowOperation cycle
0812 06:37:11.507841 (+   159us) tablet.cc:1383] finished ApplyRowOperation cycle
0812 06:37:11.507868 (+    27us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0812 06:37:11.507872 (+     4us) write_op.cc:312] APPLY: finished
0812 06:37:11.507920 (+    48us) log.cc:844] Serialized 77 byte log entry
0812 06:37:11.507961 (+    41us) write_op.cc:489] Releasing partition, row and schema locks
0812 06:37:11.507972 (+    11us) write_op.cc:454] Released schema lock
0812 06:37:11.507976 (+     4us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":1273,"num_ops":10,"prepare.queue_time_us":3199,"prepare.run_cpu_time_us":525,"prepare.run_wall_time_us":979,"raft.queue_time_us":27,"raft.run_cpu_time_us":4,"raft.run_wall_time_us":5,"replication_time_us":8098832,"thread_start_us":75,"threads_started":1}]]}
I20260812 06:37:11.508358 10529 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:42408 (ReqId={client: 3067f3aef2a44cc4bb7f2ccd6acf83db, seq_no=3316, attempt_no=0}) took 8105 ms. Trace:
I20260812 06:37:11.508406 10529 rpcz_store.cc:276] 0812 06:37:03.402952 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 06:37:03.403526 (+   574us) service_pool.cc:224] Handling call
0812 06:37:11.508350 (+8104824us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0812 06:37:03.407908 (+     0us) write_op.cc:183] PREPARE: starting on tablet 7065bb403e3047dd9a0853c49318ef1f
0812 06:37:03.407987 (+    79us) write_op.cc:432] Acquiring schema lock in shared mode
0812 06:37:03.407990 (+     3us) write_op.cc:435] Acquired schema lock
0812 06:37:03.407992 (+     2us) tablet.cc:662] Decoding operations
0812 06:37:03.408106 (+   114us) write_op.cc:620] Acquiring the partition lock for write op
0812 06:37:03.408112 (+     6us) write_op.cc:641] Partition lock acquired for write op
0812 06:37:03.408114 (+     2us) tablet.cc:685] Acquiring locks for 10 operations
0812 06:37:03.408174 (+    60us) tablet.cc:701] Row locks acquired
0812 06:37:03.408175 (+     1us) write_op.cc:260] PREPARE: finished
0812 06:37:03.408201 (+    26us) write_op.cc:270] Start()
0812 06:37:03.408219 (+    18us) write_op.cc:276] Timestamp: P: 1786516623408177 usec, L: 0
0812 06:37:03.408221 (+     2us) op_driver.cc:348] REPLICATION: starting
0812 06:37:03.408316 (+    95us) log.cc:844] Serialized 686 byte log entry
0812 06:37:11.506452 (+8098136us) op_driver.cc:464] REPLICATION: finished
0812 06:37:11.507975 (+  1523us) write_op.cc:301] APPLY: starting
0812 06:37:11.507983 (+     8us) tablet.cc:1367] starting BulkCheckPresence
0812 06:37:11.508007 (+    24us) tablet.cc:1370] finished BulkCheckPresence
0812 06:37:11.508017 (+    10us) tablet.cc:1372] starting ApplyRowOperation cycle
0812 06:37:11.508188 (+   171us) tablet.cc:1383] finished ApplyRowOperation cycle
0812 06:37:11.508211 (+    23us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0812 06:37:11.508215 (+     4us) write_op.cc:312] APPLY: finished
0812 06:37:11.508260 (+    45us) log.cc:844] Serialized 77 byte log entry
0812 06:37:11.508291 (+    31us) write_op.cc:489] Releasing partition, row and schema locks
0812 06:37:11.508301 (+    10us) write_op.cc:454] Released schema lock
0812 06:37:11.508305 (+     4us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"spinlock_wait_cycles":6528,"child_traces":[["op",{"apply.queue_time_us":1497,"num_ops":10,"prepare.queue_time_us":4189,"prepare.run_cpu_time_us":486,"prepare.run_wall_time_us":684,"raft.queue_time_us":40,"raft.run_cpu_time_us":5,"raft.run_wall_time_us":6,"replication_time_us":8098216}]]}
I20260812 06:37:11.508760 10528 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:42408 (ReqId={client: 3067f3aef2a44cc4bb7f2ccd6acf83db, seq_no=3319, attempt_no=0}) took 8105 ms. Trace:
I20260812 06:37:11.508832 10528 rpcz_store.cc:276] 0812 06:37:03.403111 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 06:37:03.403729 (+   618us) service_pool.cc:224] Handling call
0812 06:37:11.508751 (+8105022us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0812 06:37:03.408623 (+     0us) write_op.cc:183] PREPARE: starting on tablet 7065bb403e3047dd9a0853c49318ef1f
0812 06:37:03.408695 (+    72us) write_op.cc:432] Acquiring schema lock in shared mode
0812 06:37:03.408698 (+     3us) write_op.cc:435] Acquired schema lock
0812 06:37:03.408698 (+     0us) tablet.cc:662] Decoding operations
0812 06:37:03.408808 (+   110us) write_op.cc:620] Acquiring the partition lock for write op
0812 06:37:03.408813 (+     5us) write_op.cc:641] Partition lock acquired for write op
0812 06:37:03.408816 (+     3us) tablet.cc:685] Acquiring locks for 10 operations
0812 06:37:03.408884 (+    68us) tablet.cc:701] Row locks acquired
0812 06:37:03.408886 (+     2us) write_op.cc:260] PREPARE: finished
0812 06:37:03.408913 (+    27us) write_op.cc:270] Start()
0812 06:37:03.408941 (+    28us) write_op.cc:276] Timestamp: P: 1786516623408889 usec, L: 0
0812 06:37:03.408943 (+     2us) op_driver.cc:348] REPLICATION: starting
0812 06:37:03.409029 (+    86us) log.cc:844] Serialized 696 byte log entry
0812 06:37:11.506486 (+8097457us) op_driver.cc:464] REPLICATION: finished
0812 06:37:11.508377 (+  1891us) write_op.cc:301] APPLY: starting
0812 06:37:11.508387 (+    10us) tablet.cc:1367] starting BulkCheckPresence
0812 06:37:11.508415 (+    28us) tablet.cc:1370] finished BulkCheckPresence
0812 06:37:11.508427 (+    12us) tablet.cc:1372] starting ApplyRowOperation cycle
0812 06:37:11.508585 (+   158us) tablet.cc:1383] finished ApplyRowOperation cycle
0812 06:37:11.508612 (+    27us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0812 06:37:11.508616 (+     4us) write_op.cc:312] APPLY: finished
0812 06:37:11.508665 (+    49us) log.cc:844] Serialized 77 byte log entry
0812 06:37:11.508695 (+    30us) write_op.cc:489] Releasing partition, row and schema locks
0812 06:37:11.508705 (+    10us) write_op.cc:454] Released schema lock
0812 06:37:11.508709 (+     4us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":1865,"mutex_wait_us":26,"num_ops":10,"prepare.queue_time_us":4637,"prepare.run_cpu_time_us":528,"prepare.run_wall_time_us":605,"raft.queue_time_us":78,"raft.run_cpu_time_us":14,"raft.run_wall_time_us":14,"replication_time_us":8097532,"spinlock_wait_cycles":4864}]]}
I20260812 06:37:11.509102 10529 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:42408 (ReqId={client: 3067f3aef2a44cc4bb7f2ccd6acf83db, seq_no=3320, attempt_no=0}) took 8105 ms. Trace:
I20260812 06:37:11.509156 10529 rpcz_store.cc:276] 0812 06:37:03.403159 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 06:37:03.403209 (+    50us) service_pool.cc:224] Handling call
0812 06:37:11.509094 (+8105885us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0812 06:37:03.409242 (+     0us) write_op.cc:183] PREPARE: starting on tablet 7065bb403e3047dd9a0853c49318ef1f
0812 06:37:03.409314 (+    72us) write_op.cc:432] Acquiring schema lock in shared mode
0812 06:37:03.409317 (+     3us) write_op.cc:435] Acquired schema lock
0812 06:37:03.409319 (+     2us) tablet.cc:662] Decoding operations
0812 06:37:03.409423 (+   104us) write_op.cc:620] Acquiring the partition lock for write op
0812 06:37:03.409430 (+     7us) write_op.cc:641] Partition lock acquired for write op
0812 06:37:03.409433 (+     3us) tablet.cc:685] Acquiring locks for 10 operations
0812 06:37:03.409492 (+    59us) tablet.cc:701] Row locks acquired
0812 06:37:03.409494 (+     2us) write_op.cc:260] PREPARE: finished
0812 06:37:03.409523 (+    29us) write_op.cc:270] Start()
0812 06:37:03.409544 (+    21us) write_op.cc:276] Timestamp: P: 1786516623409498 usec, L: 0
0812 06:37:03.409546 (+     2us) op_driver.cc:348] REPLICATION: starting
0812 06:37:03.409658 (+   112us) log.cc:844] Serialized 686 byte log entry
0812 06:37:11.506512 (+8096854us) op_driver.cc:464] REPLICATION: finished
0812 06:37:11.508709 (+  2197us) write_op.cc:301] APPLY: starting
0812 06:37:11.508716 (+     7us) tablet.cc:1367] starting BulkCheckPresence
0812 06:37:11.508740 (+    24us) tablet.cc:1370] finished BulkCheckPresence
0812 06:37:11.508750 (+    10us) tablet.cc:1372] starting ApplyRowOperation cycle
0812 06:37:11.508927 (+   177us) tablet.cc:1383] finished ApplyRowOperation cycle
0812 06:37:11.508951 (+    24us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0812 06:37:11.508955 (+     4us) write_op.cc:312] APPLY: finished
0812 06:37:11.509003 (+    48us) log.cc:844] Serialized 77 byte log entry
0812 06:37:11.509033 (+    30us) write_op.cc:489] Releasing partition, row and schema locks
0812 06:37:11.509046 (+    13us) write_op.cc:454] Released schema lock
0812 06:37:11.509050 (+     4us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"spinlock_wait_cycles":7552,"child_traces":[["op",{"apply.queue_time_us":2174,"mutex_wait_us":715,"num_ops":10,"prepare.queue_time_us":5764,"prepare.run_cpu_time_us":493,"prepare.run_wall_time_us":536,"raft.queue_time_us":31,"raft.run_cpu_time_us":4,"raft.run_wall_time_us":4,"replication_time_us":8096955}]]}
I20260812 06:37:11.517601  6236 raft_consensus.cc:1275] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b [term 22 FOLLOWER]: Refusing update from remote peer 2df4308c852240859ada53d216ab69ae: Log matching property violated. Preceding OpId in replica: term: 21 index: 3331. Preceding OpId from leader: term: 22 index: 3339. (index mismatch)
I20260812 06:37:11.519481  6046 raft_consensus.cc:1275] T 7065bb403e3047dd9a0853c49318ef1f P ec6c8d818e744cdbb6b0d15b321010e0 [term 22 FOLLOWER]: Refusing update from remote peer 2df4308c852240859ada53d216ab69ae: Log matching property violated. Preceding OpId in replica: term: 21 index: 3272. Preceding OpId from leader: term: 22 index: 3339. (index mismatch)
I20260812 06:37:11.521890 10855 consensus_queue.cc:1048] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 41057 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3339, Last known committed idx: 3271, Time since last communication: 0.000s
I20260812 06:37:11.522716 10694 raft_consensus.cc:1275] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [term 22 FOLLOWER]: Refusing update from remote peer 2df4308c852240859ada53d216ab69ae: Log matching property violated. Preceding OpId in replica: term: 20 index: 3028. Preceding OpId from leader: term: 22 index: 3339. (index mismatch)
I20260812 06:37:11.522938 10905 consensus_queue.cc:1048] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "8d90d61da62549208fe117b4384a469b" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 41539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3339, Last known committed idx: 3323, Time since last communication: 0.000s
I20260812 06:37:11.524757 10855 consensus_queue.cc:1048] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 40897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3339, Last known committed idx: 3020, Time since last communication: 0.000s
W20260812 06:37:11.649263 10690 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.GetConsensusState from 127.0.0.1:51126 (request call id 31) took 89 ms (client timeout 100 ms). Trace:
W20260812 06:37:11.649381 10690 rpcz_store.cc:269] 0812 06:37:11.559764 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 06:37:11.559826 (+    62us) service_pool.cc:224] Handling call
0812 06:37:11.649145 (+ 89319us) spinlock_profiling.cc:243] Waited 89 ms on lock 0x55c5c48872d4. stack: 00007f6bb2f6bd35 00007f6bb2f6c325 00007f6bb2a7882d 000055c5bc490b18 000055c5bc490b6b 00007f6bb5d8cdf1 00007f6bb5d8cd5f 00007f6bb4fcabe1 00007f6bb60b45f1 00007f6bb4715ac3 00007f6bb471be1a 00007f6bb3f9af8f 00007f6bb3f9a843 00007f6bb3f9d541 00007f6bb3f9be9e 00007f6bb3f9d8cb
0812 06:37:11.649241 (+    96us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":204617088}
I20260812 06:37:11.908849 10968 mvcc.cc:204] Tried to move back new op lower bound from 7317572122700886016 to 7317572082231889920. Current Snapshot: MvccSnapshot[applied={T|T < 7317572122700886016}]
I20260812 06:37:13.671200 10975 raft_consensus.cc:493] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 2df4308c852240859ada53d216ab69ae)
I20260812 06:37:13.671366 10975 raft_consensus.cc:515] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df4308c852240859ada53d216ab69ae" member_type: VOTER last_known_addr { host: "127.4.177.6" port: 34269 } } peers { permanent_uuid: "42895a9bcba64aa484f6509605035d02" member_type: VOTER last_known_addr { host: "127.4.177.4" port: 38181 } } peers { permanent_uuid: "222b20de5a1145c6b8a11df8374e8790" member_type: VOTER last_known_addr { host: "127.4.177.5" port: 43521 } } peers { permanent_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 40897 } } peers { permanent_uuid: "8d90d61da62549208fe117b4384a469b" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 41539 } } peers { permanent_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" member_type: VOTER last_known_addr { host: "127.4.177.7" port: 46299 } } peers { permanent_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 41057 } }
I20260812 06:37:13.672106 10975 leader_election.cc:290] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 2df4308c852240859ada53d216ab69ae (127.4.177.6:34269), 42895a9bcba64aa484f6509605035d02 (127.4.177.4:38181), 222b20de5a1145c6b8a11df8374e8790 (127.4.177.5:43521), 8d90d61da62549208fe117b4384a469b (127.4.177.2:41539), b037d17b21f8422fa5e41b70eb4dcd08 (127.4.177.7:46299), ec6c8d818e744cdbb6b0d15b321010e0 (127.4.177.1:41057)
I20260812 06:37:13.672710  7320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" candidate_term: 23 candidate_status { last_received { term: 22 index: 3509 } } ignore_live_leader: false dest_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" is_pre_election: true
I20260812 06:37:13.672739  9421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" candidate_term: 23 candidate_status { last_received { term: 22 index: 3509 } } ignore_live_leader: false dest_uuid: "222b20de5a1145c6b8a11df8374e8790" is_pre_election: true
I20260812 06:37:13.672839  6046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" candidate_term: 23 candidate_status { last_received { term: 22 index: 3509 } } ignore_live_leader: false dest_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" is_pre_election: true
I20260812 06:37:13.672834  6700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" candidate_term: 23 candidate_status { last_received { term: 22 index: 3509 } } ignore_live_leader: false dest_uuid: "42895a9bcba64aa484f6509605035d02" is_pre_election: true
I20260812 06:37:13.673223  6238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" candidate_term: 23 candidate_status { last_received { term: 22 index: 3509 } } ignore_live_leader: false dest_uuid: "8d90d61da62549208fe117b4384a469b" is_pre_election: true
I20260812 06:37:13.673535 10588 leader_election.cc:304] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 5 responses out of 7 voters: 1 yes votes; 4 no votes. yes voters: 7962f587ccd842ad8a3f6ed9a5d7707e; no voters: 222b20de5a1145c6b8a11df8374e8790, 42895a9bcba64aa484f6509605035d02, b037d17b21f8422fa5e41b70eb4dcd08, ec6c8d818e744cdbb6b0d15b321010e0
I20260812 06:37:13.673722 10975 raft_consensus.cc:2749] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260812 06:37:13.680720 10977 raft_consensus.cc:493] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 2df4308c852240859ada53d216ab69ae)
I20260812 06:37:13.680845 10977 raft_consensus.cc:515] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df4308c852240859ada53d216ab69ae" member_type: VOTER last_known_addr { host: "127.4.177.6" port: 34269 } } peers { permanent_uuid: "42895a9bcba64aa484f6509605035d02" member_type: VOTER last_known_addr { host: "127.4.177.4" port: 38181 } } peers { permanent_uuid: "222b20de5a1145c6b8a11df8374e8790" member_type: VOTER last_known_addr { host: "127.4.177.5" port: 43521 } } peers { permanent_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 40897 } } peers { permanent_uuid: "8d90d61da62549208fe117b4384a469b" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 41539 } } peers { permanent_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" member_type: VOTER last_known_addr { host: "127.4.177.7" port: 46299 } } peers { permanent_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 41057 } }
I20260812 06:37:13.681720 10977 leader_election.cc:290] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 2df4308c852240859ada53d216ab69ae (127.4.177.6:34269), 42895a9bcba64aa484f6509605035d02 (127.4.177.4:38181), 7962f587ccd842ad8a3f6ed9a5d7707e (127.4.177.3:40897), 8d90d61da62549208fe117b4384a469b (127.4.177.2:41539), b037d17b21f8422fa5e41b70eb4dcd08 (127.4.177.7:46299), ec6c8d818e744cdbb6b0d15b321010e0 (127.4.177.1:41057)
I20260812 06:37:13.681756 10978 raft_consensus.cc:493] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 2df4308c852240859ada53d216ab69ae)
I20260812 06:37:13.682163 10690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "222b20de5a1145c6b8a11df8374e8790" candidate_term: 23 candidate_status { last_received { term: 22 index: 3513 } } ignore_live_leader: false dest_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" is_pre_election: true
I20260812 06:37:13.682158  6700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "222b20de5a1145c6b8a11df8374e8790" candidate_term: 23 candidate_status { last_received { term: 22 index: 3513 } } ignore_live_leader: false dest_uuid: "42895a9bcba64aa484f6509605035d02" is_pre_election: true
I20260812 06:37:13.682322 10690 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 222b20de5a1145c6b8a11df8374e8790 in term 22.
I20260812 06:37:13.682133 10978 raft_consensus.cc:515] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df4308c852240859ada53d216ab69ae" member_type: VOTER last_known_addr { host: "127.4.177.6" port: 34269 } } peers { permanent_uuid: "42895a9bcba64aa484f6509605035d02" member_type: VOTER last_known_addr { host: "127.4.177.4" port: 38181 } } peers { permanent_uuid: "222b20de5a1145c6b8a11df8374e8790" member_type: VOTER last_known_addr { host: "127.4.177.5" port: 43521 } } peers { permanent_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 40897 } } peers { permanent_uuid: "8d90d61da62549208fe117b4384a469b" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 41539 } } peers { permanent_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" member_type: VOTER last_known_addr { host: "127.4.177.7" port: 46299 } } peers { permanent_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 41057 } }
I20260812 06:37:13.682333  6046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "222b20de5a1145c6b8a11df8374e8790" candidate_term: 23 candidate_status { last_received { term: 22 index: 3513 } } ignore_live_leader: false dest_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" is_pre_election: true
I20260812 06:37:13.682390  7320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "222b20de5a1145c6b8a11df8374e8790" candidate_term: 23 candidate_status { last_received { term: 22 index: 3513 } } ignore_live_leader: false dest_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" is_pre_election: true
I20260812 06:37:13.682497  6700 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 222b20de5a1145c6b8a11df8374e8790 in term 22.
I20260812 06:37:13.682977  6238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "222b20de5a1145c6b8a11df8374e8790" candidate_term: 23 candidate_status { last_received { term: 22 index: 3513 } } ignore_live_leader: false dest_uuid: "8d90d61da62549208fe117b4384a469b" is_pre_election: true
I20260812 06:37:13.683616 10978 leader_election.cc:290] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 2df4308c852240859ada53d216ab69ae (127.4.177.6:34269), 222b20de5a1145c6b8a11df8374e8790 (127.4.177.5:43521), 7962f587ccd842ad8a3f6ed9a5d7707e (127.4.177.3:40897), 8d90d61da62549208fe117b4384a469b (127.4.177.2:41539), b037d17b21f8422fa5e41b70eb4dcd08 (127.4.177.7:46299), ec6c8d818e744cdbb6b0d15b321010e0 (127.4.177.1:41057)
I20260812 06:37:13.683876  9421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "42895a9bcba64aa484f6509605035d02" candidate_term: 23 candidate_status { last_received { term: 22 index: 3513 } } ignore_live_leader: false dest_uuid: "222b20de5a1145c6b8a11df8374e8790" is_pre_election: true
I20260812 06:37:13.683951  6238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "42895a9bcba64aa484f6509605035d02" candidate_term: 23 candidate_status { last_received { term: 22 index: 3513 } } ignore_live_leader: false dest_uuid: "8d90d61da62549208fe117b4384a469b" is_pre_election: true
I20260812 06:37:13.684012 10690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "42895a9bcba64aa484f6509605035d02" candidate_term: 23 candidate_status { last_received { term: 22 index: 3513 } } ignore_live_leader: false dest_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" is_pre_election: true
I20260812 06:37:13.684038  9421 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 42895a9bcba64aa484f6509605035d02 in term 22.
I20260812 06:37:13.684161 10690 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 42895a9bcba64aa484f6509605035d02 in term 22.
I20260812 06:37:13.684410  7320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "42895a9bcba64aa484f6509605035d02" candidate_term: 23 candidate_status { last_received { term: 22 index: 3513 } } ignore_live_leader: false dest_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" is_pre_election: true
I20260812 06:37:13.684435  6046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "42895a9bcba64aa484f6509605035d02" candidate_term: 23 candidate_status { last_received { term: 22 index: 3513 } } ignore_live_leader: false dest_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" is_pre_election: true
I20260812 06:37:14.018911 10979 raft_consensus.cc:493] T 7065bb403e3047dd9a0853c49318ef1f P ec6c8d818e744cdbb6b0d15b321010e0 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 2df4308c852240859ada53d216ab69ae)
I20260812 06:37:14.019132 10979 raft_consensus.cc:515] T 7065bb403e3047dd9a0853c49318ef1f P ec6c8d818e744cdbb6b0d15b321010e0 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df4308c852240859ada53d216ab69ae" member_type: VOTER last_known_addr { host: "127.4.177.6" port: 34269 } } peers { permanent_uuid: "42895a9bcba64aa484f6509605035d02" member_type: VOTER last_known_addr { host: "127.4.177.4" port: 38181 } } peers { permanent_uuid: "222b20de5a1145c6b8a11df8374e8790" member_type: VOTER last_known_addr { host: "127.4.177.5" port: 43521 } } peers { permanent_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 40897 } } peers { permanent_uuid: "8d90d61da62549208fe117b4384a469b" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 41539 } } peers { permanent_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" member_type: VOTER last_known_addr { host: "127.4.177.7" port: 46299 } } peers { permanent_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 41057 } }
I20260812 06:37:14.019863 10979 leader_election.cc:290] T 7065bb403e3047dd9a0853c49318ef1f P ec6c8d818e744cdbb6b0d15b321010e0 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 2df4308c852240859ada53d216ab69ae (127.4.177.6:34269), 42895a9bcba64aa484f6509605035d02 (127.4.177.4:38181), 222b20de5a1145c6b8a11df8374e8790 (127.4.177.5:43521), 7962f587ccd842ad8a3f6ed9a5d7707e (127.4.177.3:40897), 8d90d61da62549208fe117b4384a469b (127.4.177.2:41539), b037d17b21f8422fa5e41b70eb4dcd08 (127.4.177.7:46299)
I20260812 06:37:14.020380  7320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" candidate_term: 23 candidate_status { last_received { term: 22 index: 3406 } } ignore_live_leader: false dest_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" is_pre_election: true
I20260812 06:37:14.020401  9421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" candidate_term: 23 candidate_status { last_received { term: 22 index: 3406 } } ignore_live_leader: false dest_uuid: "222b20de5a1145c6b8a11df8374e8790" is_pre_election: true
I20260812 06:37:14.020399  6700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" candidate_term: 23 candidate_status { last_received { term: 22 index: 3406 } } ignore_live_leader: false dest_uuid: "42895a9bcba64aa484f6509605035d02" is_pre_election: true
I20260812 06:37:14.020572  7320 raft_consensus.cc:2410] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ec6c8d818e744cdbb6b0d15b321010e0 for term 23 because replica has last-logged OpId of term: 22 index: 3425, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 3406.
I20260812 06:37:14.020581  9421 raft_consensus.cc:2410] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ec6c8d818e744cdbb6b0d15b321010e0 for term 23 because replica has last-logged OpId of term: 22 index: 3513, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 3406.
I20260812 06:37:14.020602  6700 raft_consensus.cc:2410] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ec6c8d818e744cdbb6b0d15b321010e0 for term 23 because replica has last-logged OpId of term: 22 index: 3513, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 3406.
I20260812 06:37:14.020766  6238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" candidate_term: 23 candidate_status { last_received { term: 22 index: 3406 } } ignore_live_leader: false dest_uuid: "8d90d61da62549208fe117b4384a469b" is_pre_election: true
I20260812 06:37:14.021143 10980 raft_consensus.cc:493] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 2df4308c852240859ada53d216ab69ae)
I20260812 06:37:14.021178 10690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" candidate_term: 23 candidate_status { last_received { term: 22 index: 3406 } } ignore_live_leader: false dest_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" is_pre_election: true
I20260812 06:37:14.021256 10980 raft_consensus.cc:515] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df4308c852240859ada53d216ab69ae" member_type: VOTER last_known_addr { host: "127.4.177.6" port: 34269 } } peers { permanent_uuid: "42895a9bcba64aa484f6509605035d02" member_type: VOTER last_known_addr { host: "127.4.177.4" port: 38181 } } peers { permanent_uuid: "222b20de5a1145c6b8a11df8374e8790" member_type: VOTER last_known_addr { host: "127.4.177.5" port: 43521 } } peers { permanent_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 40897 } } peers { permanent_uuid: "8d90d61da62549208fe117b4384a469b" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 41539 } } peers { permanent_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" member_type: VOTER last_known_addr { host: "127.4.177.7" port: 46299 } } peers { permanent_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 41057 } }
I20260812 06:37:14.021287  5945 leader_election.cc:304] T 7065bb403e3047dd9a0853c49318ef1f P ec6c8d818e744cdbb6b0d15b321010e0 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 5 responses out of 7 voters: 1 yes votes; 4 no votes. yes voters: ec6c8d818e744cdbb6b0d15b321010e0; no voters: 222b20de5a1145c6b8a11df8374e8790, 42895a9bcba64aa484f6509605035d02, 8d90d61da62549208fe117b4384a469b, b037d17b21f8422fa5e41b70eb4dcd08
I20260812 06:37:14.021364 10690 raft_consensus.cc:2410] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ec6c8d818e744cdbb6b0d15b321010e0 for term 23 because replica has last-logged OpId of term: 22 index: 3509, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 3406.
I20260812 06:37:14.021471 10979 raft_consensus.cc:2749] T 7065bb403e3047dd9a0853c49318ef1f P ec6c8d818e744cdbb6b0d15b321010e0 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260812 06:37:14.021922 10980 leader_election.cc:290] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 2df4308c852240859ada53d216ab69ae (127.4.177.6:34269), 42895a9bcba64aa484f6509605035d02 (127.4.177.4:38181), 222b20de5a1145c6b8a11df8374e8790 (127.4.177.5:43521), 7962f587ccd842ad8a3f6ed9a5d7707e (127.4.177.3:40897), 8d90d61da62549208fe117b4384a469b (127.4.177.2:41539), ec6c8d818e744cdbb6b0d15b321010e0 (127.4.177.1:41057)
I20260812 06:37:14.022336  9421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" candidate_term: 23 candidate_status { last_received { term: 22 index: 3425 } } ignore_live_leader: false dest_uuid: "222b20de5a1145c6b8a11df8374e8790" is_pre_election: true
I20260812 06:37:14.022396 10690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" candidate_term: 23 candidate_status { last_received { term: 22 index: 3425 } } ignore_live_leader: false dest_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" is_pre_election: true
I20260812 06:37:14.022495  9421 raft_consensus.cc:2410] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b037d17b21f8422fa5e41b70eb4dcd08 for term 23 because replica has last-logged OpId of term: 22 index: 3513, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 3425.
I20260812 06:37:14.022531 10690 raft_consensus.cc:2410] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b037d17b21f8422fa5e41b70eb4dcd08 for term 23 because replica has last-logged OpId of term: 22 index: 3509, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 3425.
I20260812 06:37:14.022532  6700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" candidate_term: 23 candidate_status { last_received { term: 22 index: 3425 } } ignore_live_leader: false dest_uuid: "42895a9bcba64aa484f6509605035d02" is_pre_election: true
I20260812 06:37:14.022713  6700 raft_consensus.cc:2410] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b037d17b21f8422fa5e41b70eb4dcd08 for term 23 because replica has last-logged OpId of term: 22 index: 3513, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 3425.
I20260812 06:37:14.022760  6046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" candidate_term: 23 candidate_status { last_received { term: 22 index: 3425 } } ignore_live_leader: false dest_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" is_pre_election: true
I20260812 06:37:14.022887  6046 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P ec6c8d818e744cdbb6b0d15b321010e0 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b037d17b21f8422fa5e41b70eb4dcd08 in term 22.
I20260812 06:37:14.023026  6238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" candidate_term: 23 candidate_status { last_received { term: 22 index: 3425 } } ignore_live_leader: false dest_uuid: "8d90d61da62549208fe117b4384a469b" is_pre_election: true
I20260812 06:37:14.023406  7211 leader_election.cc:304] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 6 responses out of 7 voters: 2 yes votes; 4 no votes. yes voters: b037d17b21f8422fa5e41b70eb4dcd08, ec6c8d818e744cdbb6b0d15b321010e0; no voters: 222b20de5a1145c6b8a11df8374e8790, 42895a9bcba64aa484f6509605035d02, 7962f587ccd842ad8a3f6ed9a5d7707e, 8d90d61da62549208fe117b4384a469b
I20260812 06:37:14.023592 10980 raft_consensus.cc:2749] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260812 06:37:14.024629 10981 raft_consensus.cc:493] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 2df4308c852240859ada53d216ab69ae)
I20260812 06:37:14.024757 10981 raft_consensus.cc:515] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df4308c852240859ada53d216ab69ae" member_type: VOTER last_known_addr { host: "127.4.177.6" port: 34269 } } peers { permanent_uuid: "42895a9bcba64aa484f6509605035d02" member_type: VOTER last_known_addr { host: "127.4.177.4" port: 38181 } } peers { permanent_uuid: "222b20de5a1145c6b8a11df8374e8790" member_type: VOTER last_known_addr { host: "127.4.177.5" port: 43521 } } peers { permanent_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 40897 } } peers { permanent_uuid: "8d90d61da62549208fe117b4384a469b" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 41539 } } peers { permanent_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" member_type: VOTER last_known_addr { host: "127.4.177.7" port: 46299 } } peers { permanent_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 41057 } }
I20260812 06:37:14.025347 10981 leader_election.cc:290] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 2df4308c852240859ada53d216ab69ae (127.4.177.6:34269), 42895a9bcba64aa484f6509605035d02 (127.4.177.4:38181), 222b20de5a1145c6b8a11df8374e8790 (127.4.177.5:43521), 7962f587ccd842ad8a3f6ed9a5d7707e (127.4.177.3:40897), b037d17b21f8422fa5e41b70eb4dcd08 (127.4.177.7:46299), ec6c8d818e744cdbb6b0d15b321010e0 (127.4.177.1:41057)
I20260812 06:37:14.025696  6700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "8d90d61da62549208fe117b4384a469b" candidate_term: 23 candidate_status { last_received { term: 22 index: 3466 } } ignore_live_leader: false dest_uuid: "42895a9bcba64aa484f6509605035d02" is_pre_election: true
I20260812 06:37:14.025777  7320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "8d90d61da62549208fe117b4384a469b" candidate_term: 23 candidate_status { last_received { term: 22 index: 3466 } } ignore_live_leader: false dest_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" is_pre_election: true
I20260812 06:37:14.025780 10690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "8d90d61da62549208fe117b4384a469b" candidate_term: 23 candidate_status { last_received { term: 22 index: 3466 } } ignore_live_leader: false dest_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" is_pre_election: true
I20260812 06:37:14.025840  6700 raft_consensus.cc:2410] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8d90d61da62549208fe117b4384a469b for term 23 because replica has last-logged OpId of term: 22 index: 3513, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 3466.
I20260812 06:37:14.025846  9421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "8d90d61da62549208fe117b4384a469b" candidate_term: 23 candidate_status { last_received { term: 22 index: 3466 } } ignore_live_leader: false dest_uuid: "222b20de5a1145c6b8a11df8374e8790" is_pre_election: true
I20260812 06:37:14.025898  7320 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8d90d61da62549208fe117b4384a469b in term 22.
I20260812 06:37:14.025925 10690 raft_consensus.cc:2410] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8d90d61da62549208fe117b4384a469b for term 23 because replica has last-logged OpId of term: 22 index: 3509, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 3466.
I20260812 06:37:14.025974  9421 raft_consensus.cc:2410] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8d90d61da62549208fe117b4384a469b for term 23 because replica has last-logged OpId of term: 22 index: 3513, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 3466.
I20260812 06:37:14.026417  6046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "8d90d61da62549208fe117b4384a469b" candidate_term: 23 candidate_status { last_received { term: 22 index: 3466 } } ignore_live_leader: false dest_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" is_pre_election: true
I20260812 06:37:14.026553  6046 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P ec6c8d818e744cdbb6b0d15b321010e0 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8d90d61da62549208fe117b4384a469b in term 22.
W20260812 06:37:15.556499 10588 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.4.177.6:34269 timed out after 1.883s (SENT) [suppressed 1 similar messages]
W20260812 06:37:15.569217  7211 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.4.177.6:34269 timed out after 1.546s (SENT) [suppressed 1 similar messages]
W20260812 06:37:15.608155  9315 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.4.177.6:34269 timed out after 1.925s (SENT)
W20260812 06:37:15.661125  6597 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.4.177.6:34269 timed out after 1.976s (SENT) [suppressed 4 similar messages]
I20260812 06:37:15.724223 10990 raft_consensus.cc:493] T 7065bb403e3047dd9a0853c49318ef1f P ec6c8d818e744cdbb6b0d15b321010e0 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 2df4308c852240859ada53d216ab69ae)
I20260812 06:37:15.724364 10990 raft_consensus.cc:515] T 7065bb403e3047dd9a0853c49318ef1f P ec6c8d818e744cdbb6b0d15b321010e0 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df4308c852240859ada53d216ab69ae" member_type: VOTER last_known_addr { host: "127.4.177.6" port: 34269 } } peers { permanent_uuid: "42895a9bcba64aa484f6509605035d02" member_type: VOTER last_known_addr { host: "127.4.177.4" port: 38181 } } peers { permanent_uuid: "222b20de5a1145c6b8a11df8374e8790" member_type: VOTER last_known_addr { host: "127.4.177.5" port: 43521 } } peers { permanent_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 40897 } } peers { permanent_uuid: "8d90d61da62549208fe117b4384a469b" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 41539 } } peers { permanent_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" member_type: VOTER last_known_addr { host: "127.4.177.7" port: 46299 } } peers { permanent_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 41057 } }
I20260812 06:37:15.725162 10990 leader_election.cc:290] T 7065bb403e3047dd9a0853c49318ef1f P ec6c8d818e744cdbb6b0d15b321010e0 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 2df4308c852240859ada53d216ab69ae (127.4.177.6:34269), 42895a9bcba64aa484f6509605035d02 (127.4.177.4:38181), 222b20de5a1145c6b8a11df8374e8790 (127.4.177.5:43521), 7962f587ccd842ad8a3f6ed9a5d7707e (127.4.177.3:40897), 8d90d61da62549208fe117b4384a469b (127.4.177.2:41539), b037d17b21f8422fa5e41b70eb4dcd08 (127.4.177.7:46299)
I20260812 06:37:15.725664  7320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" candidate_term: 23 candidate_status { last_received { term: 22 index: 3406 } } ignore_live_leader: false dest_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" is_pre_election: true
I20260812 06:37:15.725664  6700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" candidate_term: 23 candidate_status { last_received { term: 22 index: 3406 } } ignore_live_leader: false dest_uuid: "42895a9bcba64aa484f6509605035d02" is_pre_election: true
I20260812 06:37:15.725704  6238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" candidate_term: 23 candidate_status { last_received { term: 22 index: 3406 } } ignore_live_leader: false dest_uuid: "8d90d61da62549208fe117b4384a469b" is_pre_election: true
I20260812 06:37:15.725709  9421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" candidate_term: 23 candidate_status { last_received { term: 22 index: 3406 } } ignore_live_leader: false dest_uuid: "222b20de5a1145c6b8a11df8374e8790" is_pre_election: true
I20260812 06:37:15.725862  6238 raft_consensus.cc:2410] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ec6c8d818e744cdbb6b0d15b321010e0 for term 23 because replica has last-logged OpId of term: 22 index: 3466, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 3406.
I20260812 06:37:15.725863  6700 raft_consensus.cc:2410] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ec6c8d818e744cdbb6b0d15b321010e0 for term 23 because replica has last-logged OpId of term: 22 index: 3513, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 3406.
I20260812 06:37:15.725872  7320 raft_consensus.cc:2410] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ec6c8d818e744cdbb6b0d15b321010e0 for term 23 because replica has last-logged OpId of term: 22 index: 3425, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 3406.
I20260812 06:37:15.725880  9421 raft_consensus.cc:2410] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ec6c8d818e744cdbb6b0d15b321010e0 for term 23 because replica has last-logged OpId of term: 22 index: 3513, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 3406.
I20260812 06:37:15.726406  5945 leader_election.cc:304] T 7065bb403e3047dd9a0853c49318ef1f P ec6c8d818e744cdbb6b0d15b321010e0 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 5 responses out of 7 voters: 1 yes votes; 4 no votes. yes voters: ec6c8d818e744cdbb6b0d15b321010e0; no voters: 222b20de5a1145c6b8a11df8374e8790, 42895a9bcba64aa484f6509605035d02, 8d90d61da62549208fe117b4384a469b, b037d17b21f8422fa5e41b70eb4dcd08
I20260812 06:37:15.726542 10690 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" candidate_term: 23 candidate_status { last_received { term: 22 index: 3406 } } ignore_live_leader: false dest_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" is_pre_election: true
I20260812 06:37:15.726641 10990 raft_consensus.cc:2749] T 7065bb403e3047dd9a0853c49318ef1f P ec6c8d818e744cdbb6b0d15b321010e0 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260812 06:37:15.726771 10690 raft_consensus.cc:2410] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ec6c8d818e744cdbb6b0d15b321010e0 for term 23 because replica has last-logged OpId of term: 22 index: 3509, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 3406.
W20260812 06:37:15.792347  5945 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.4.177.6:34269 timed out after 1.771s (SENT)
W20260812 06:37:15.858558  6150 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.4.177.6:34269 timed out after 1.832s (SENT)
I20260812 06:37:15.878250 11001 raft_consensus.cc:493] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 2df4308c852240859ada53d216ab69ae)
I20260812 06:37:15.878410 11001 raft_consensus.cc:515] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df4308c852240859ada53d216ab69ae" member_type: VOTER last_known_addr { host: "127.4.177.6" port: 34269 } } peers { permanent_uuid: "42895a9bcba64aa484f6509605035d02" member_type: VOTER last_known_addr { host: "127.4.177.4" port: 38181 } } peers { permanent_uuid: "222b20de5a1145c6b8a11df8374e8790" member_type: VOTER last_known_addr { host: "127.4.177.5" port: 43521 } } peers { permanent_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 40897 } } peers { permanent_uuid: "8d90d61da62549208fe117b4384a469b" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 41539 } } peers { permanent_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" member_type: VOTER last_known_addr { host: "127.4.177.7" port: 46299 } } peers { permanent_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 41057 } }
I20260812 06:37:15.879190 11001 leader_election.cc:290] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 2df4308c852240859ada53d216ab69ae (127.4.177.6:34269), 42895a9bcba64aa484f6509605035d02 (127.4.177.4:38181), 222b20de5a1145c6b8a11df8374e8790 (127.4.177.5:43521), 8d90d61da62549208fe117b4384a469b (127.4.177.2:41539), b037d17b21f8422fa5e41b70eb4dcd08 (127.4.177.7:46299), ec6c8d818e744cdbb6b0d15b321010e0 (127.4.177.1:41057)
I20260812 06:37:15.879612  6700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" candidate_term: 23 candidate_status { last_received { term: 22 index: 3509 } } ignore_live_leader: false dest_uuid: "42895a9bcba64aa484f6509605035d02" is_pre_election: true
I20260812 06:37:15.879814  6700 raft_consensus.cc:2410] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7962f587ccd842ad8a3f6ed9a5d7707e for term 23 because replica has last-logged OpId of term: 22 index: 3513, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 3509.
I20260812 06:37:15.879797  7320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" candidate_term: 23 candidate_status { last_received { term: 22 index: 3509 } } ignore_live_leader: false dest_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" is_pre_election: true
I20260812 06:37:15.879822  6238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" candidate_term: 23 candidate_status { last_received { term: 22 index: 3509 } } ignore_live_leader: false dest_uuid: "8d90d61da62549208fe117b4384a469b" is_pre_election: true
I20260812 06:37:15.879930  7320 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7962f587ccd842ad8a3f6ed9a5d7707e in term 22.
I20260812 06:37:15.879933  9421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" candidate_term: 23 candidate_status { last_received { term: 22 index: 3509 } } ignore_live_leader: false dest_uuid: "222b20de5a1145c6b8a11df8374e8790" is_pre_election: true
I20260812 06:37:15.879977  6238 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7962f587ccd842ad8a3f6ed9a5d7707e in term 22.
I20260812 06:37:15.880105  9421 raft_consensus.cc:2410] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7962f587ccd842ad8a3f6ed9a5d7707e for term 23 because replica has last-logged OpId of term: 22 index: 3513, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 3509.
I20260812 06:37:15.880405  6046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" candidate_term: 23 candidate_status { last_received { term: 22 index: 3509 } } ignore_live_leader: false dest_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" is_pre_election: true
I20260812 06:37:15.880549  6046 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P ec6c8d818e744cdbb6b0d15b321010e0 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7962f587ccd842ad8a3f6ed9a5d7707e in term 22.
I20260812 06:37:15.880873 10588 leader_election.cc:304] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 6 responses out of 7 voters: 4 yes votes; 2 no votes. yes voters: 7962f587ccd842ad8a3f6ed9a5d7707e, 8d90d61da62549208fe117b4384a469b, b037d17b21f8422fa5e41b70eb4dcd08, ec6c8d818e744cdbb6b0d15b321010e0; no voters: 222b20de5a1145c6b8a11df8374e8790, 42895a9bcba64aa484f6509605035d02
I20260812 06:37:15.881196 11001 raft_consensus.cc:2804] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260812 06:37:15.881263 11001 raft_consensus.cc:493] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [term 22 FOLLOWER]: Starting leader election (detected failure of leader 2df4308c852240859ada53d216ab69ae)
I20260812 06:37:15.881295 11001 raft_consensus.cc:3060] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [term 22 FOLLOWER]: Advancing to term 23
I20260812 06:37:15.882398 11001 raft_consensus.cc:515] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df4308c852240859ada53d216ab69ae" member_type: VOTER last_known_addr { host: "127.4.177.6" port: 34269 } } peers { permanent_uuid: "42895a9bcba64aa484f6509605035d02" member_type: VOTER last_known_addr { host: "127.4.177.4" port: 38181 } } peers { permanent_uuid: "222b20de5a1145c6b8a11df8374e8790" member_type: VOTER last_known_addr { host: "127.4.177.5" port: 43521 } } peers { permanent_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 40897 } } peers { permanent_uuid: "8d90d61da62549208fe117b4384a469b" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 41539 } } peers { permanent_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" member_type: VOTER last_known_addr { host: "127.4.177.7" port: 46299 } } peers { permanent_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 41057 } }
I20260812 06:37:15.883090 11001 leader_election.cc:290] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [CANDIDATE]: Term 23 election: Requested vote from peers 2df4308c852240859ada53d216ab69ae (127.4.177.6:34269), 42895a9bcba64aa484f6509605035d02 (127.4.177.4:38181), 222b20de5a1145c6b8a11df8374e8790 (127.4.177.5:43521), 8d90d61da62549208fe117b4384a469b (127.4.177.2:41539), b037d17b21f8422fa5e41b70eb4dcd08 (127.4.177.7:46299), ec6c8d818e744cdbb6b0d15b321010e0 (127.4.177.1:41057)
I20260812 06:37:15.883430  6700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" candidate_term: 23 candidate_status { last_received { term: 22 index: 3509 } } ignore_live_leader: false dest_uuid: "42895a9bcba64aa484f6509605035d02"
I20260812 06:37:15.883575  6700 raft_consensus.cc:3060] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [term 22 FOLLOWER]: Advancing to term 23
I20260812 06:37:15.883569  6046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" candidate_term: 23 candidate_status { last_received { term: 22 index: 3509 } } ignore_live_leader: false dest_uuid: "ec6c8d818e744cdbb6b0d15b321010e0"
I20260812 06:37:15.883589  7320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" candidate_term: 23 candidate_status { last_received { term: 22 index: 3509 } } ignore_live_leader: false dest_uuid: "b037d17b21f8422fa5e41b70eb4dcd08"
I20260812 06:37:15.883705  7320 raft_consensus.cc:3060] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08 [term 22 FOLLOWER]: Advancing to term 23
I20260812 06:37:15.883706  6046 raft_consensus.cc:3060] T 7065bb403e3047dd9a0853c49318ef1f P ec6c8d818e744cdbb6b0d15b321010e0 [term 22 FOLLOWER]: Advancing to term 23
I20260812 06:37:15.883775  6238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" candidate_term: 23 candidate_status { last_received { term: 22 index: 3509 } } ignore_live_leader: false dest_uuid: "8d90d61da62549208fe117b4384a469b"
I20260812 06:37:15.883896  6238 raft_consensus.cc:3060] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b [term 22 FOLLOWER]: Advancing to term 23
I20260812 06:37:15.884745  7320 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7962f587ccd842ad8a3f6ed9a5d7707e in term 23.
I20260812 06:37:15.884804  6700 raft_consensus.cc:2410] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 7962f587ccd842ad8a3f6ed9a5d7707e for term 23 because replica has last-logged OpId of term: 22 index: 3513, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 3509.
I20260812 06:37:15.884774  6046 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P ec6c8d818e744cdbb6b0d15b321010e0 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7962f587ccd842ad8a3f6ed9a5d7707e in term 23.
I20260812 06:37:15.884944  6238 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7962f587ccd842ad8a3f6ed9a5d7707e in term 23.
I20260812 06:37:15.885079  9421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" candidate_term: 23 candidate_status { last_received { term: 22 index: 3509 } } ignore_live_leader: false dest_uuid: "222b20de5a1145c6b8a11df8374e8790"
I20260812 06:37:15.885241  9421 raft_consensus.cc:3060] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [term 22 FOLLOWER]: Advancing to term 23
I20260812 06:37:15.885239 10588 leader_election.cc:304] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 7 voters: 4 yes votes; 0 no votes. yes voters: 7962f587ccd842ad8a3f6ed9a5d7707e, 8d90d61da62549208fe117b4384a469b, b037d17b21f8422fa5e41b70eb4dcd08, ec6c8d818e744cdbb6b0d15b321010e0; no voters: 
I20260812 06:37:15.885654 11001 raft_consensus.cc:2804] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [term 23 FOLLOWER]: Leader election won for term 23
I20260812 06:37:15.885923 11001 raft_consensus.cc:697] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [term 23 LEADER]: Becoming Leader. State: Replica: 7962f587ccd842ad8a3f6ed9a5d7707e, State: Running, Role: LEADER
I20260812 06:37:15.886436  9421 raft_consensus.cc:2410] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 7962f587ccd842ad8a3f6ed9a5d7707e for term 23 because replica has last-logged OpId of term: 22 index: 3513, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 3509.
I20260812 06:37:15.886343 11001 consensus_queue.cc:237] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3502, Committed index: 3502, Last appended: 22.3509, Last appended by leader: 3509, Current term: 23, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df4308c852240859ada53d216ab69ae" member_type: VOTER last_known_addr { host: "127.4.177.6" port: 34269 } } peers { permanent_uuid: "42895a9bcba64aa484f6509605035d02" member_type: VOTER last_known_addr { host: "127.4.177.4" port: 38181 } } peers { permanent_uuid: "222b20de5a1145c6b8a11df8374e8790" member_type: VOTER last_known_addr { host: "127.4.177.5" port: 43521 } } peers { permanent_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 40897 } } peers { permanent_uuid: "8d90d61da62549208fe117b4384a469b" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 41539 } } peers { permanent_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" member_type: VOTER last_known_addr { host: "127.4.177.7" port: 46299 } } peers { permanent_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 41057 } }
I20260812 06:37:15.889384  5808 catalog_manager.cc:5719] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e reported cstate change: term changed from 22 to 23, leader changed from 2df4308c852240859ada53d216ab69ae (127.4.177.6) to 7962f587ccd842ad8a3f6ed9a5d7707e (127.4.177.3). New cstate: current_term: 23 leader_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df4308c852240859ada53d216ab69ae" member_type: VOTER last_known_addr { host: "127.4.177.6" port: 34269 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "42895a9bcba64aa484f6509605035d02" member_type: VOTER last_known_addr { host: "127.4.177.4" port: 38181 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "222b20de5a1145c6b8a11df8374e8790" member_type: VOTER last_known_addr { host: "127.4.177.5" port: 43521 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 40897 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8d90d61da62549208fe117b4384a469b" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 41539 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" member_type: VOTER last_known_addr { host: "127.4.177.7" port: 46299 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 41057 } health_report { overall_health: UNKNOWN } } }
W20260812 06:37:16.094451  9714 connection.cc:256] RPC call timeout handler was delayed by -3.3543429709970951s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1s for a response
W20260812 06:37:16.094563  9714 connection.cc:256] RPC call timeout handler was delayed by -3.1319296651054174s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1s for a response
W20260812 06:37:16.094607  9714 connection.cc:256] RPC call timeout handler was delayed by -3.0251924709882587s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1s for a response
W20260812 06:37:16.094645  9710 connection.cc:256] RPC call timeout handler was delayed by -3.2451970190741122s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1s for a response
I20260812 06:37:16.095444  9797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "222b20de5a1145c6b8a11df8374e8790" candidate_term: 23 candidate_status { last_received { term: 22 index: 3513 } } ignore_live_leader: false dest_uuid: "2df4308c852240859ada53d216ab69ae" is_pre_election: true
I20260812 06:37:16.095494  9794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" candidate_term: 23 candidate_status { last_received { term: 22 index: 3509 } } ignore_live_leader: false dest_uuid: "2df4308c852240859ada53d216ab69ae"
I20260812 06:37:16.095790  9796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" candidate_term: 23 candidate_status { last_received { term: 22 index: 3509 } } ignore_live_leader: false dest_uuid: "2df4308c852240859ada53d216ab69ae" is_pre_election: true
I20260812 06:37:16.095979  9798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "8d90d61da62549208fe117b4384a469b" candidate_term: 23 candidate_status { last_received { term: 22 index: 3466 } } ignore_live_leader: false dest_uuid: "2df4308c852240859ada53d216ab69ae" is_pre_election: true
I20260812 06:37:16.096176  9799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "42895a9bcba64aa484f6509605035d02" candidate_term: 23 candidate_status { last_received { term: 22 index: 3513 } } ignore_live_leader: false dest_uuid: "2df4308c852240859ada53d216ab69ae" is_pre_election: true
I20260812 06:37:16.096371  9800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "222b20de5a1145c6b8a11df8374e8790" candidate_term: 23 candidate_status { last_received { term: 22 index: 3513 } } ignore_live_leader: false dest_uuid: "2df4308c852240859ada53d216ab69ae" is_pre_election: true
I20260812 06:37:16.096587  9802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "42895a9bcba64aa484f6509605035d02" candidate_term: 23 candidate_status { last_received { term: 22 index: 3513 } } ignore_live_leader: false dest_uuid: "2df4308c852240859ada53d216ab69ae" is_pre_election: true
I20260812 06:37:16.096796  9804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "8d90d61da62549208fe117b4384a469b" candidate_term: 23 candidate_status { last_received { term: 22 index: 3466 } } ignore_live_leader: false dest_uuid: "2df4308c852240859ada53d216ab69ae" is_pre_election: true
I20260812 06:37:16.096928  9315 leader_election.cc:304] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 7 responses out of 7 voters: 3 yes votes; 4 no votes. yes voters: 222b20de5a1145c6b8a11df8374e8790, 42895a9bcba64aa484f6509605035d02, 7962f587ccd842ad8a3f6ed9a5d7707e; no voters: 2df4308c852240859ada53d216ab69ae, 8d90d61da62549208fe117b4384a469b, b037d17b21f8422fa5e41b70eb4dcd08, ec6c8d818e744cdbb6b0d15b321010e0
I20260812 06:37:16.097000  9806 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" candidate_term: 23 candidate_status { last_received { term: 22 index: 3425 } } ignore_live_leader: false dest_uuid: "2df4308c852240859ada53d216ab69ae" is_pre_election: true
I20260812 06:37:16.097232  9807 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" candidate_term: 23 candidate_status { last_received { term: 22 index: 3425 } } ignore_live_leader: false dest_uuid: "2df4308c852240859ada53d216ab69ae" is_pre_election: true
I20260812 06:37:16.097467  9808 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" candidate_term: 23 candidate_status { last_received { term: 22 index: 3509 } } ignore_live_leader: false dest_uuid: "2df4308c852240859ada53d216ab69ae" is_pre_election: true
I20260812 06:37:16.097640  6150 leader_election.cc:304] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 7 responses out of 7 voters: 3 yes votes; 4 no votes. yes voters: 8d90d61da62549208fe117b4384a469b, b037d17b21f8422fa5e41b70eb4dcd08, ec6c8d818e744cdbb6b0d15b321010e0; no voters: 222b20de5a1145c6b8a11df8374e8790, 2df4308c852240859ada53d216ab69ae, 42895a9bcba64aa484f6509605035d02, 7962f587ccd842ad8a3f6ed9a5d7707e
I20260812 06:37:16.097713  9809 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" candidate_term: 23 candidate_status { last_received { term: 22 index: 3406 } } ignore_live_leader: false dest_uuid: "2df4308c852240859ada53d216ab69ae" is_pre_election: true
I20260812 06:37:16.097915  9810 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" candidate_term: 23 candidate_status { last_received { term: 22 index: 3406 } } ignore_live_leader: false dest_uuid: "2df4308c852240859ada53d216ab69ae" is_pre_election: true
I20260812 06:37:16.098002 10690 raft_consensus.cc:1240] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [term 23 LEADER]: Rejecting Update request from peer 2df4308c852240859ada53d216ab69ae for earlier term 22. Current term is 23. Ops: [22.3510-22.3513]
I20260812 06:37:16.098114  9814 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" candidate_term: 23 candidate_status { last_received { term: 22 index: 3406 } } ignore_live_leader: false dest_uuid: "2df4308c852240859ada53d216ab69ae" is_pre_election: true
I20260812 06:37:16.098310  9815 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" candidate_term: 23 candidate_status { last_received { term: 22 index: 3509 } } ignore_live_leader: false dest_uuid: "2df4308c852240859ada53d216ab69ae" is_pre_election: true
I20260812 06:37:16.100046 11017 raft_consensus.cc:2749] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260812 06:37:16.100255 11013 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:41528 (ReqId={client: 3067f3aef2a44cc4bb7f2ccd6acf83db, seq_no=3490, attempt_no=0}) took 3936 ms. Trace:
I20260812 06:37:16.100338 11011 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:41528 (ReqId={client: 3067f3aef2a44cc4bb7f2ccd6acf83db, seq_no=3489, attempt_no=0}) took 3939 ms. Trace:
I20260812 06:37:16.100507  6238 raft_consensus.cc:1240] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b [term 23 FOLLOWER]: Rejecting Update request from peer 2df4308c852240859ada53d216ab69ae for earlier term 22. Current term is 23. Ops: [22.3467-22.3513]
I20260812 06:37:16.100406 11011 rpcz_store.cc:276] 0812 06:37:12.160829 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 06:37:12.161631 (+   802us) service_pool.cc:224] Handling call
0812 06:37:16.100317 (+3938686us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0812 06:37:12.164160 (+     0us) write_op.cc:183] PREPARE: starting on tablet 7065bb403e3047dd9a0853c49318ef1f
0812 06:37:12.164396 (+   236us) write_op.cc:432] Acquiring schema lock in shared mode
0812 06:37:12.164399 (+     3us) write_op.cc:435] Acquired schema lock
0812 06:37:12.164400 (+     1us) tablet.cc:662] Decoding operations
0812 06:37:12.164785 (+   385us) write_op.cc:620] Acquiring the partition lock for write op
0812 06:37:12.164791 (+     6us) write_op.cc:641] Partition lock acquired for write op
0812 06:37:12.164794 (+     3us) tablet.cc:685] Acquiring locks for 10 operations
0812 06:37:12.164859 (+    65us) tablet.cc:701] Row locks acquired
0812 06:37:12.164861 (+     2us) write_op.cc:260] PREPARE: finished
0812 06:37:12.164898 (+    37us) write_op.cc:270] Start()
0812 06:37:12.164920 (+    22us) write_op.cc:276] Timestamp: P: 1786516632164850 usec, L: 0
0812 06:37:12.164922 (+     2us) op_driver.cc:348] REPLICATION: starting
0812 06:37:12.165025 (+   103us) log.cc:844] Serialized 696 byte log entry
0812 06:37:16.097380 (+3932355us) op_driver.cc:464] REPLICATION: finished
0812 06:37:16.099555 (+  2175us) write_op.cc:301] APPLY: starting
0812 06:37:16.099579 (+    24us) tablet.cc:1367] starting BulkCheckPresence
0812 06:37:16.099613 (+    34us) tablet.cc:1370] finished BulkCheckPresence
0812 06:37:16.099627 (+    14us) tablet.cc:1372] starting ApplyRowOperation cycle
0812 06:37:16.099803 (+   176us) tablet.cc:1383] finished ApplyRowOperation cycle
0812 06:37:16.099831 (+    28us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0812 06:37:16.099836 (+     5us) write_op.cc:312] APPLY: finished
0812 06:37:16.099914 (+    78us) log.cc:844] Serialized 77 byte log entry
0812 06:37:16.100253 (+   339us) write_op.cc:489] Releasing partition, row and schema locks
0812 06:37:16.100264 (+    11us) write_op.cc:454] Released schema lock
0812 06:37:16.100272 (+     8us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":2128,"mutex_wait_us":288,"num_ops":10,"prepare.queue_time_us":2270,"prepare.run_cpu_time_us":490,"prepare.run_wall_time_us":917,"replication_time_us":3932443,"thread_start_us":46,"threads_started":1}]]}
I20260812 06:37:16.100891 11016 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:41528 (ReqId={client: 3067f3aef2a44cc4bb7f2ccd6acf83db, seq_no=3491, attempt_no=0}) took 3936 ms. Trace:
I20260812 06:37:16.100946 11016 rpcz_store.cc:276] 0812 06:37:12.164432 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 06:37:12.164470 (+    38us) service_pool.cc:224] Handling call
0812 06:37:16.100866 (+3936396us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0812 06:37:12.165940 (+     0us) write_op.cc:183] PREPARE: starting on tablet 7065bb403e3047dd9a0853c49318ef1f
0812 06:37:12.166004 (+    64us) write_op.cc:432] Acquiring schema lock in shared mode
0812 06:37:12.166007 (+     3us) write_op.cc:435] Acquired schema lock
0812 06:37:12.166009 (+     2us) tablet.cc:662] Decoding operations
0812 06:37:12.166088 (+    79us) write_op.cc:620] Acquiring the partition lock for write op
0812 06:37:12.166093 (+     5us) write_op.cc:641] Partition lock acquired for write op
0812 06:37:12.166096 (+     3us) tablet.cc:685] Acquiring locks for 10 operations
0812 06:37:12.166157 (+    61us) tablet.cc:701] Row locks acquired
0812 06:37:12.166159 (+     2us) write_op.cc:260] PREPARE: finished
0812 06:37:12.166184 (+    25us) write_op.cc:270] Start()
0812 06:37:12.166200 (+    16us) write_op.cc:276] Timestamp: P: 1786516632166136 usec, L: 0
0812 06:37:12.166202 (+     2us) op_driver.cc:348] REPLICATION: starting
0812 06:37:12.166322 (+   120us) log.cc:844] Serialized 686 byte log entry
0812 06:37:16.097596 (+3931274us) op_driver.cc:464] REPLICATION: finished
0812 06:37:16.100443 (+  2847us) write_op.cc:301] APPLY: starting
0812 06:37:16.100454 (+    11us) tablet.cc:1367] starting BulkCheckPresence
0812 06:37:16.100482 (+    28us) tablet.cc:1370] finished BulkCheckPresence
0812 06:37:16.100494 (+    12us) tablet.cc:1372] starting ApplyRowOperation cycle
0812 06:37:16.100657 (+   163us) tablet.cc:1383] finished ApplyRowOperation cycle
0812 06:37:16.100688 (+    31us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0812 06:37:16.100693 (+     5us) write_op.cc:312] APPLY: finished
0812 06:37:16.100761 (+    68us) log.cc:844] Serialized 77 byte log entry
0812 06:37:16.100796 (+    35us) write_op.cc:489] Releasing partition, row and schema locks
0812 06:37:16.100809 (+    13us) write_op.cc:454] Released schema lock
0812 06:37:16.100815 (+     6us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":2810,"num_ops":10,"prepare.queue_time_us":1216,"prepare.run_cpu_time_us":430,"prepare.run_wall_time_us":430,"replication_time_us":3931379,"thread_start_us":51,"threads_started":1}]]}
I20260812 06:37:16.100310 11013 rpcz_store.cc:276] 0812 06:37:12.163671 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 06:37:12.163710 (+    39us) service_pool.cc:224] Handling call
0812 06:37:16.100229 (+3936519us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0812 06:37:12.165104 (+     0us) write_op.cc:183] PREPARE: starting on tablet 7065bb403e3047dd9a0853c49318ef1f
0812 06:37:12.165171 (+    67us) write_op.cc:432] Acquiring schema lock in shared mode
0812 06:37:12.165174 (+     3us) write_op.cc:435] Acquired schema lock
0812 06:37:12.165176 (+     2us) tablet.cc:662] Decoding operations
0812 06:37:12.165251 (+    75us) write_op.cc:620] Acquiring the partition lock for write op
0812 06:37:12.165255 (+     4us) write_op.cc:641] Partition lock acquired for write op
0812 06:37:12.165256 (+     1us) tablet.cc:685] Acquiring locks for 10 operations
0812 06:37:12.165346 (+    90us) tablet.cc:701] Row locks acquired
0812 06:37:12.165349 (+     3us) write_op.cc:260] PREPARE: finished
0812 06:37:12.165375 (+    26us) write_op.cc:270] Start()
0812 06:37:12.165393 (+    18us) write_op.cc:276] Timestamp: P: 1786516632165327 usec, L: 0
0812 06:37:12.165395 (+     2us) op_driver.cc:348] REPLICATION: starting
0812 06:37:12.165845 (+   450us) log.cc:844] Serialized 696 byte log entry
0812 06:37:16.097481 (+3931636us) op_driver.cc:464] REPLICATION: finished
0812 06:37:16.099633 (+  2152us) write_op.cc:301] APPLY: starting
0812 06:37:16.099641 (+     8us) tablet.cc:1367] starting BulkCheckPresence
0812 06:37:16.099663 (+    22us) tablet.cc:1370] finished BulkCheckPresence
0812 06:37:16.099672 (+     9us) tablet.cc:1372] starting ApplyRowOperation cycle
0812 06:37:16.099835 (+   163us) tablet.cc:1383] finished ApplyRowOperation cycle
0812 06:37:16.099853 (+    18us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0812 06:37:16.099856 (+     3us) write_op.cc:312] APPLY: finished
0812 06:37:16.099914 (+    58us) log.cc:844] Serialized 77 byte log entry
0812 06:37:16.100123 (+   209us) write_op.cc:489] Releasing partition, row and schema locks
0812 06:37:16.100143 (+    20us) write_op.cc:454] Released schema lock
0812 06:37:16.100151 (+     8us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":2115,"num_ops":10,"prepare.queue_time_us":1126,"prepare.run_cpu_time_us":461,"prepare.run_wall_time_us":808,"replication_time_us":3932070,"spinlock_wait_cycles":384,"thread_start_us":184,"threads_started":2}]]}
I20260812 06:37:16.101603 11012 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:41528 (ReqId={client: 3067f3aef2a44cc4bb7f2ccd6acf83db, seq_no=3488, attempt_no=0}) took 3944 ms. Trace:
I20260812 06:37:16.101670 10956 consensus_queue.cc:1059] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 40897 }, Status: INVALID_TERM, Last received: 22.3509, Next index: 3510, Last known committed idx: 3502, Time since last communication: 0.000s
I20260812 06:37:16.101781 10956 consensus_queue.cc:1243] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [LEADER]: Peer 7962f587ccd842ad8a3f6ed9a5d7707e log is divergent from this leader: its last log entry 23.3510 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3502
I20260812 06:37:16.101658 11012 rpcz_store.cc:276] 0812 06:37:12.157352 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 06:37:12.158419 (+  1067us) service_pool.cc:224] Handling call
0812 06:37:16.101569 (+3943150us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0812 06:37:12.158863 (+     0us) write_op.cc:183] PREPARE: starting on tablet 7065bb403e3047dd9a0853c49318ef1f
0812 06:37:12.158941 (+    78us) write_op.cc:432] Acquiring schema lock in shared mode
0812 06:37:12.158943 (+     2us) write_op.cc:435] Acquired schema lock
0812 06:37:12.158945 (+     2us) tablet.cc:662] Decoding operations
0812 06:37:12.159024 (+    79us) write_op.cc:620] Acquiring the partition lock for write op
0812 06:37:12.159029 (+     5us) write_op.cc:641] Partition lock acquired for write op
0812 06:37:12.159032 (+     3us) tablet.cc:685] Acquiring locks for 10 operations
0812 06:37:12.159090 (+    58us) tablet.cc:701] Row locks acquired
0812 06:37:12.159091 (+     1us) write_op.cc:260] PREPARE: finished
0812 06:37:12.159117 (+    26us) write_op.cc:270] Start()
0812 06:37:12.159138 (+    21us) write_op.cc:276] Timestamp: P: 1786516632159069 usec, L: 0
0812 06:37:12.159139 (+     1us) op_driver.cc:348] REPLICATION: starting
0812 06:37:12.159533 (+   394us) log.cc:844] Serialized 696 byte log entry
0812 06:37:16.097227 (+3937694us) op_driver.cc:464] REPLICATION: finished
0812 06:37:16.099090 (+  1863us) write_op.cc:301] APPLY: starting
0812 06:37:16.099110 (+    20us) tablet.cc:1367] starting BulkCheckPresence
0812 06:37:16.099157 (+    47us) tablet.cc:1370] finished BulkCheckPresence
0812 06:37:16.099170 (+    13us) tablet.cc:1372] starting ApplyRowOperation cycle
0812 06:37:16.099410 (+   240us) tablet.cc:1383] finished ApplyRowOperation cycle
0812 06:37:16.101197 (+  1787us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0812 06:37:16.101202 (+     5us) write_op.cc:312] APPLY: finished
0812 06:37:16.101359 (+   157us) log.cc:844] Serialized 77 byte log entry
0812 06:37:16.101477 (+   118us) write_op.cc:489] Releasing partition, row and schema locks
0812 06:37:16.101492 (+    15us) write_op.cc:454] Released schema lock
0812 06:37:16.101500 (+     8us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":1792,"num_ops":10,"prepare.queue_time_us":200,"prepare.run_cpu_time_us":529,"prepare.run_wall_time_us":721,"replication_time_us":3938055,"spinlock_wait_cycles":666880,"thread_start_us":77,"threads_started":1}]]}
I20260812 06:37:16.101833 10956 consensus_queue.cc:1059] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8d90d61da62549208fe117b4384a469b" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 41539 }, Status: INVALID_TERM, Last received: 22.3466, Next index: 3467, Last known committed idx: 3462, Time since last communication: 0.000s
I20260812 06:37:16.101924 10956 raft_consensus.cc:3055] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [term 22 LEADER]: Stepping down as leader of term 22
I20260812 06:37:16.101961 10956 raft_consensus.cc:740] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 2df4308c852240859ada53d216ab69ae, State: Running, Role: LEADER
I20260812 06:37:16.102092 11010 raft_consensus.cc:2749] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260812 06:37:16.102142 10956 consensus_queue.cc:260] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3509, Committed index: 3509, Last appended: 22.3513, Last appended by leader: 3513, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df4308c852240859ada53d216ab69ae" member_type: VOTER last_known_addr { host: "127.4.177.6" port: 34269 } } peers { permanent_uuid: "42895a9bcba64aa484f6509605035d02" member_type: VOTER last_known_addr { host: "127.4.177.4" port: 38181 } } peers { permanent_uuid: "222b20de5a1145c6b8a11df8374e8790" member_type: VOTER last_known_addr { host: "127.4.177.5" port: 43521 } } peers { permanent_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 40897 } } peers { permanent_uuid: "8d90d61da62549208fe117b4384a469b" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 41539 } } peers { permanent_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" member_type: VOTER last_known_addr { host: "127.4.177.7" port: 46299 } } peers { permanent_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 41057 } }
I20260812 06:37:16.102421 10956 raft_consensus.cc:3060] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [term 22 FOLLOWER]: Advancing to term 23
W20260812 06:37:16.104712  9744 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:41528: Illegal state: replica 2df4308c852240859ada53d216ab69ae is not leader of this config: current role FOLLOWER
W20260812 06:37:16.104727  9742 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:41528: Illegal state: replica 2df4308c852240859ada53d216ab69ae is not leader of this config: current role FOLLOWER
W20260812 06:37:16.105237  9738 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:41528: Illegal state: replica 2df4308c852240859ada53d216ab69ae is not leader of this config: current role FOLLOWER
I20260812 06:37:16.106003  6046 raft_consensus.cc:1240] T 7065bb403e3047dd9a0853c49318ef1f P ec6c8d818e744cdbb6b0d15b321010e0 [term 23 FOLLOWER]: Rejecting Update request from peer 2df4308c852240859ada53d216ab69ae for earlier term 22. Current term is 23. Ops: [22.3407-22.3513]
I20260812 06:37:16.106709  6700 raft_consensus.cc:1240] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [term 23 FOLLOWER]: Rejecting Update request from peer 2df4308c852240859ada53d216ab69ae for earlier term 22. Current term is 23. Ops: []
I20260812 06:37:16.107722  6597 leader_election.cc:304] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 7 responses out of 7 voters: 3 yes votes; 4 no votes. yes voters: 222b20de5a1145c6b8a11df8374e8790, 42895a9bcba64aa484f6509605035d02, 7962f587ccd842ad8a3f6ed9a5d7707e; no voters: 2df4308c852240859ada53d216ab69ae, 8d90d61da62549208fe117b4384a469b, b037d17b21f8422fa5e41b70eb4dcd08, ec6c8d818e744cdbb6b0d15b321010e0
I20260812 06:37:16.109314  9421 raft_consensus.cc:1240] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [term 23 FOLLOWER]: Rejecting Update request from peer 2df4308c852240859ada53d216ab69ae for earlier term 22. Current term is 23. Ops: []
I20260812 06:37:16.109982  7320 raft_consensus.cc:1240] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08 [term 23 FOLLOWER]: Rejecting Update request from peer 2df4308c852240859ada53d216ab69ae for earlier term 22. Current term is 23. Ops: [22.3426-22.3513]
W20260812 06:37:16.110072  9738 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:41528: Illegal state: replica 2df4308c852240859ada53d216ab69ae is not leader of this config: current role FOLLOWER
W20260812 06:37:16.110702  6625 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:42408: Illegal state: replica 42895a9bcba64aa484f6509605035d02 is not leader of this config: current role FOLLOWER
W20260812 06:37:16.110702  6631 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:42408: Illegal state: replica 42895a9bcba64aa484f6509605035d02 is not leader of this config: current role FOLLOWER
W20260812 06:37:16.111332  6623 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:42408: Illegal state: replica 42895a9bcba64aa484f6509605035d02 is not leader of this config: current role FOLLOWER
I20260812 06:37:16.112094 11020 raft_consensus.cc:2749] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260812 06:37:16.113795  6623 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:42408: Illegal state: replica 42895a9bcba64aa484f6509605035d02 is not leader of this config: current role FOLLOWER
W20260812 06:37:16.116072  9359 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:46864: Illegal state: replica 222b20de5a1145c6b8a11df8374e8790 is not leader of this config: current role FOLLOWER
W20260812 06:37:16.117188  9359 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:46864: Illegal state: replica 222b20de5a1145c6b8a11df8374e8790 is not leader of this config: current role FOLLOWER
W20260812 06:37:16.118122  9359 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:46864: Illegal state: replica 222b20de5a1145c6b8a11df8374e8790 is not leader of this config: current role FOLLOWER
W20260812 06:37:16.119796  9359 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:46864: Illegal state: replica 222b20de5a1145c6b8a11df8374e8790 is not leader of this config: current role FOLLOWER
W20260812 06:37:16.125360  6185 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:42054: Illegal state: replica 8d90d61da62549208fe117b4384a469b is not leader of this config: current role FOLLOWER
W20260812 06:37:16.125341  6179 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:42054: Illegal state: replica 8d90d61da62549208fe117b4384a469b is not leader of this config: current role FOLLOWER
W20260812 06:37:16.125341  6178 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:42054: Illegal state: replica 8d90d61da62549208fe117b4384a469b is not leader of this config: current role FOLLOWER
W20260812 06:37:16.127904  6179 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:42054: Illegal state: replica 8d90d61da62549208fe117b4384a469b is not leader of this config: current role FOLLOWER
W20260812 06:37:16.132409  7249 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:58454: Illegal state: replica b037d17b21f8422fa5e41b70eb4dcd08 is not leader of this config: current role FOLLOWER
W20260812 06:37:16.133572  7249 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:58454: Illegal state: replica b037d17b21f8422fa5e41b70eb4dcd08 is not leader of this config: current role FOLLOWER
W20260812 06:37:16.133680  7254 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:58454: Illegal state: replica b037d17b21f8422fa5e41b70eb4dcd08 is not leader of this config: current role FOLLOWER
W20260812 06:37:16.137948  7254 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:58454: Illegal state: replica b037d17b21f8422fa5e41b70eb4dcd08 is not leader of this config: current role FOLLOWER
W20260812 06:37:16.140352  5988 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:42334: Illegal state: replica ec6c8d818e744cdbb6b0d15b321010e0 is not leader of this config: current role FOLLOWER
W20260812 06:37:16.140350  5984 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:42334: Illegal state: replica ec6c8d818e744cdbb6b0d15b321010e0 is not leader of this config: current role FOLLOWER
W20260812 06:37:16.142614  5984 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:42334: Illegal state: replica ec6c8d818e744cdbb6b0d15b321010e0 is not leader of this config: current role FOLLOWER
W20260812 06:37:16.144649  5984 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:42334: Illegal state: replica ec6c8d818e744cdbb6b0d15b321010e0 is not leader of this config: current role FOLLOWER
I20260812 06:37:16.159193  7320 raft_consensus.cc:1275] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08 [term 23 FOLLOWER]: Refusing update from remote peer 7962f587ccd842ad8a3f6ed9a5d7707e: Log matching property violated. Preceding OpId in replica: term: 22 index: 3425. Preceding OpId from leader: term: 23 index: 3511. (index mismatch)
I20260812 06:37:16.159441  6238 raft_consensus.cc:1275] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b [term 23 FOLLOWER]: Refusing update from remote peer 7962f587ccd842ad8a3f6ed9a5d7707e: Log matching property violated. Preceding OpId in replica: term: 22 index: 3466. Preceding OpId from leader: term: 23 index: 3511. (index mismatch)
I20260812 06:37:16.159690  9794 raft_consensus.cc:1275] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [term 23 FOLLOWER]: Refusing update from remote peer 7962f587ccd842ad8a3f6ed9a5d7707e: Log matching property violated. Preceding OpId in replica: term: 22 index: 3513. Preceding OpId from leader: term: 23 index: 3511. (term mismatch)
I20260812 06:37:16.159780  9794 pending_rounds.cc:85] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae: Aborting all ops after (but not including) 3510
I20260812 06:37:16.159812  9794 pending_rounds.cc:107] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae: Aborting uncommitted WRITE_OP operation due to leader change: 22.3511
I20260812 06:37:16.159797  6046 raft_consensus.cc:1275] T 7065bb403e3047dd9a0853c49318ef1f P ec6c8d818e744cdbb6b0d15b321010e0 [term 23 FOLLOWER]: Refusing update from remote peer 7962f587ccd842ad8a3f6ed9a5d7707e: Log matching property violated. Preceding OpId in replica: term: 22 index: 3406. Preceding OpId from leader: term: 23 index: 3511. (index mismatch)
W20260812 06:37:16.159950  9794 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:41528: Aborted: Op aborted by new leader
I20260812 06:37:16.160112  9794 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:41528 (ReqId={client: 3067f3aef2a44cc4bb7f2ccd6acf83db, seq_no=3494, attempt_no=0}) took 3985 ms. Trace:
I20260812 06:37:16.160245  6700 raft_consensus.cc:1275] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [term 23 FOLLOWER]: Refusing update from remote peer 7962f587ccd842ad8a3f6ed9a5d7707e: Log matching property violated. Preceding OpId in replica: term: 22 index: 3513. Preceding OpId from leader: term: 23 index: 3511. (term mismatch)
I20260812 06:37:16.160177  9794 rpcz_store.cc:276] 0812 06:37:12.174235 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 06:37:12.174392 (+   157us) service_pool.cc:224] Handling call
0812 06:37:16.160099 (+3985707us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0812 06:37:12.175401 (+     0us) write_op.cc:183] PREPARE: starting on tablet 7065bb403e3047dd9a0853c49318ef1f
0812 06:37:12.175721 (+   320us) write_op.cc:432] Acquiring schema lock in shared mode
0812 06:37:12.175724 (+     3us) write_op.cc:435] Acquired schema lock
0812 06:37:12.175726 (+     2us) tablet.cc:662] Decoding operations
0812 06:37:12.175824 (+    98us) write_op.cc:620] Acquiring the partition lock for write op
0812 06:37:12.175832 (+     8us) write_op.cc:641] Partition lock acquired for write op
0812 06:37:12.175837 (+     5us) tablet.cc:685] Acquiring locks for 10 operations
0812 06:37:12.175900 (+    63us) tablet.cc:701] Row locks acquired
0812 06:37:12.175902 (+     2us) write_op.cc:260] PREPARE: finished
0812 06:37:12.175929 (+    27us) write_op.cc:270] Start()
0812 06:37:12.175948 (+    19us) write_op.cc:276] Timestamp: P: 1786516632175880 usec, L: 0
0812 06:37:12.175950 (+     2us) op_driver.cc:348] REPLICATION: starting
0812 06:37:12.176054 (+   104us) log.cc:844] Serialized 696 byte log entry
0812 06:37:16.159905 (+3983851us) op_driver.cc:464] REPLICATION: finished
0812 06:37:16.159913 (+     8us) op_driver.cc:389] HandleFailure(Aborted: Op aborted by new leader)
0812 06:37:16.159916 (+     3us) write_op.cc:489] Releasing partition, row and schema locks
0812 06:37:16.159940 (+    24us) write_op.cc:454] Released schema lock
0812 06:37:16.159946 (+     6us) write_op.cc:335] FINISH: Op aborted
Metrics: {"child_traces":[["op",{"num_ops":10,"prepare.queue_time_us":780,"prepare.run_cpu_time_us":469,"prepare.run_wall_time_us":918,"replication_time_us":3983896}]]}
I20260812 06:37:16.160315  6700 pending_rounds.cc:85] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02: Aborting all ops after (but not including) 3510
I20260812 06:37:16.160344  6700 pending_rounds.cc:107] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02: Aborting uncommitted WRITE_OP operation due to leader change: 22.3511
I20260812 06:37:16.160490  9794 pending_rounds.cc:107] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae: Aborting uncommitted WRITE_OP operation due to leader change: 22.3512
I20260812 06:37:16.160562  6700 pending_rounds.cc:107] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02: Aborting uncommitted WRITE_OP operation due to leader change: 22.3512
W20260812 06:37:16.160629  9794 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:41528: Aborted: Op aborted by new leader
I20260812 06:37:16.160708  6700 pending_rounds.cc:107] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02: Aborting uncommitted WRITE_OP operation due to leader change: 22.3513
I20260812 06:37:16.160738  9794 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:41528 (ReqId={client: 3067f3aef2a44cc4bb7f2ccd6acf83db, seq_no=3493, attempt_no=0}) took 3986 ms. Trace:
I20260812 06:37:16.160779  9794 rpcz_store.cc:276] 0812 06:37:12.173986 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 06:37:12.175262 (+  1276us) service_pool.cc:224] Handling call
0812 06:37:16.160730 (+3985468us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0812 06:37:12.176345 (+     0us) write_op.cc:183] PREPARE: starting on tablet 7065bb403e3047dd9a0853c49318ef1f
0812 06:37:12.176406 (+    61us) write_op.cc:432] Acquiring schema lock in shared mode
0812 06:37:12.176409 (+     3us) write_op.cc:435] Acquired schema lock
0812 06:37:12.176410 (+     1us) tablet.cc:662] Decoding operations
0812 06:37:12.176485 (+    75us) write_op.cc:620] Acquiring the partition lock for write op
0812 06:37:12.176490 (+     5us) write_op.cc:641] Partition lock acquired for write op
0812 06:37:12.176493 (+     3us) tablet.cc:685] Acquiring locks for 10 operations
0812 06:37:12.176556 (+    63us) tablet.cc:701] Row locks acquired
0812 06:37:12.176558 (+     2us) write_op.cc:260] PREPARE: finished
0812 06:37:12.176585 (+    27us) write_op.cc:270] Start()
0812 06:37:12.176605 (+    20us) write_op.cc:276] Timestamp: P: 1786516632176535 usec, L: 0
0812 06:37:12.176609 (+     4us) op_driver.cc:348] REPLICATION: starting
0812 06:37:12.176716 (+   107us) log.cc:844] Serialized 696 byte log entry
0812 06:37:16.160588 (+3983872us) op_driver.cc:464] REPLICATION: finished
0812 06:37:16.160594 (+     6us) op_driver.cc:389] HandleFailure(Aborted: Op aborted by new leader)
0812 06:37:16.160597 (+     3us) write_op.cc:489] Releasing partition, row and schema locks
0812 06:37:16.160619 (+    22us) write_op.cc:454] Released schema lock
0812 06:37:16.160625 (+     6us) write_op.cc:335] FINISH: Op aborted
Metrics: {"child_traces":[["op",{"num_ops":10,"prepare.queue_time_us":855,"prepare.run_cpu_time_us":420,"prepare.run_wall_time_us":1018,"replication_time_us":3983938}]]}
I20260812 06:37:16.161022  9794 pending_rounds.cc:107] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae: Aborting uncommitted WRITE_OP operation due to leader change: 22.3513
W20260812 06:37:16.161175  9794 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:41528: Aborted: Op aborted by new leader
I20260812 06:37:16.161295  9794 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:41528 (ReqId={client: 3067f3aef2a44cc4bb7f2ccd6acf83db, seq_no=3495, attempt_no=0}) took 3987 ms. Trace:
I20260812 06:37:16.161299 11034 consensus_queue.cc:1048] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8d90d61da62549208fe117b4384a469b" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 41539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3510, Last known committed idx: 3462, Time since last communication: 0.000s
I20260812 06:37:16.161343  9794 rpcz_store.cc:276] 0812 06:37:12.175395 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 06:37:12.175433 (+    38us) service_pool.cc:224] Handling call
0812 06:37:16.161284 (+3985851us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0812 06:37:12.177384 (+     0us) write_op.cc:183] PREPARE: starting on tablet 7065bb403e3047dd9a0853c49318ef1f
0812 06:37:12.177440 (+    56us) write_op.cc:432] Acquiring schema lock in shared mode
0812 06:37:12.177442 (+     2us) write_op.cc:435] Acquired schema lock
0812 06:37:12.177444 (+     2us) tablet.cc:662] Decoding operations
0812 06:37:12.177553 (+   109us) write_op.cc:620] Acquiring the partition lock for write op
0812 06:37:12.177558 (+     5us) write_op.cc:641] Partition lock acquired for write op
0812 06:37:12.177561 (+     3us) tablet.cc:685] Acquiring locks for 10 operations
0812 06:37:12.177620 (+    59us) tablet.cc:701] Row locks acquired
0812 06:37:12.177622 (+     2us) write_op.cc:260] PREPARE: finished
0812 06:37:12.177648 (+    26us) write_op.cc:270] Start()
0812 06:37:12.177666 (+    18us) write_op.cc:276] Timestamp: P: 1786516632177599 usec, L: 0
0812 06:37:12.177668 (+     2us) op_driver.cc:348] REPLICATION: starting
0812 06:37:12.177767 (+    99us) log.cc:844] Serialized 696 byte log entry
0812 06:37:16.161133 (+3983366us) op_driver.cc:464] REPLICATION: finished
0812 06:37:16.161139 (+     6us) op_driver.cc:389] HandleFailure(Aborted: Op aborted by new leader)
0812 06:37:16.161142 (+     3us) write_op.cc:489] Releasing partition, row and schema locks
0812 06:37:16.161164 (+    22us) write_op.cc:454] Released schema lock
0812 06:37:16.161169 (+     5us) write_op.cc:335] FINISH: Op aborted
Metrics: {"child_traces":[["op",{"num_ops":10,"prepare.queue_time_us":1748,"prepare.run_cpu_time_us":431,"prepare.run_wall_time_us":430,"replication_time_us":3983408}]]}
I20260812 06:37:16.161494 11001 consensus_queue.cc:1048] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 41057 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3510, Last known committed idx: 3398, Time since last communication: 0.000s
I20260812 06:37:16.161720 11032 consensus_queue.cc:1048] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [LEADER]: Connected to new peer: Peer: permanent_uuid: "42895a9bcba64aa484f6509605035d02" member_type: VOTER last_known_addr { host: "127.4.177.4" port: 38181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3510, Last known committed idx: 3505, Time since last communication: 0.000s
I20260812 06:37:16.161950 11032 consensus_queue.cc:1243] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [LEADER]: Peer 42895a9bcba64aa484f6509605035d02 log is divergent from this leader: its last log entry 22.3510 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3505
I20260812 06:37:16.162025  9421 raft_consensus.cc:1275] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [term 23 FOLLOWER]: Refusing update from remote peer 7962f587ccd842ad8a3f6ed9a5d7707e: Log matching property violated. Preceding OpId in replica: term: 22 index: 3513. Preceding OpId from leader: term: 23 index: 3511. (term mismatch)
I20260812 06:37:16.162098  9421 pending_rounds.cc:85] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790: Aborting all ops after (but not including) 3510
I20260812 06:37:16.162123  9421 pending_rounds.cc:107] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790: Aborting uncommitted WRITE_OP operation due to leader change: 22.3511
I20260812 06:37:16.162370  9421 pending_rounds.cc:107] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790: Aborting uncommitted WRITE_OP operation due to leader change: 22.3512
I20260812 06:37:16.162510  9421 pending_rounds.cc:107] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790: Aborting uncommitted WRITE_OP operation due to leader change: 22.3513
I20260812 06:37:16.162592 11002 consensus_queue.cc:1048] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [LEADER]: Connected to new peer: Peer: permanent_uuid: "2df4308c852240859ada53d216ab69ae" member_type: VOTER last_known_addr { host: "127.4.177.6" port: 34269 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3510, Last known committed idx: 3509, Time since last communication: 0.000s
I20260812 06:37:16.162689 11002 consensus_queue.cc:1243] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [LEADER]: Peer 2df4308c852240859ada53d216ab69ae log is divergent from this leader: its last log entry 22.3510 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3509
I20260812 06:37:16.163306  6700 raft_consensus.cc:1217] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [term 23 FOLLOWER]: Deduplicated request from leader. Original: 22.3505->[22.3506-23.3514]   Dedup: 22.3509->[23.3510-23.3514]
I20260812 06:37:16.163425  6700 pending_rounds.cc:85] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02: Aborting all ops after (but not including) 3509
I20260812 06:37:16.163460  6700 pending_rounds.cc:107] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02: Aborting uncommitted WRITE_OP operation due to leader change: 22.3510
I20260812 06:37:16.163460 11032 consensus_queue.cc:1048] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [LEADER]: Connected to new peer: Peer: permanent_uuid: "222b20de5a1145c6b8a11df8374e8790" member_type: VOTER last_known_addr { host: "127.4.177.5" port: 43521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3510, Last known committed idx: 3505, Time since last communication: 0.000s
I20260812 06:37:16.163542 11032 consensus_queue.cc:1243] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [LEADER]: Peer 222b20de5a1145c6b8a11df8374e8790 log is divergent from this leader: its last log entry 22.3510 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3505
I20260812 06:37:16.163582 11001 consensus_queue.cc:1048] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [LEADER]: Connected to new peer: Peer: permanent_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" member_type: VOTER last_known_addr { host: "127.4.177.7" port: 46299 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3510, Last known committed idx: 3424, Time since last communication: 0.000s
I20260812 06:37:16.163856  9794 pending_rounds.cc:85] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae: Aborting all ops after (but not including) 3509
I20260812 06:37:16.163906  9794 pending_rounds.cc:107] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae: Aborting uncommitted WRITE_OP operation due to leader change: 22.3510
W20260812 06:37:16.164008  9794 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:41528: Aborted: Op aborted by new leader
I20260812 06:37:16.164127  9794 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:41528 (ReqId={client: 3067f3aef2a44cc4bb7f2ccd6acf83db, seq_no=3492, attempt_no=0}) took 3990 ms. Trace:
I20260812 06:37:16.164165  9794 rpcz_store.cc:276] 0812 06:37:12.173905 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 06:37:12.173990 (+    85us) service_pool.cc:224] Handling call
0812 06:37:16.164119 (+3990129us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0812 06:37:12.174661 (+     0us) write_op.cc:183] PREPARE: starting on tablet 7065bb403e3047dd9a0853c49318ef1f
0812 06:37:12.174743 (+    82us) write_op.cc:432] Acquiring schema lock in shared mode
0812 06:37:12.174747 (+     4us) write_op.cc:435] Acquired schema lock
0812 06:37:12.174748 (+     1us) tablet.cc:662] Decoding operations
0812 06:37:12.174835 (+    87us) write_op.cc:620] Acquiring the partition lock for write op
0812 06:37:12.174840 (+     5us) write_op.cc:641] Partition lock acquired for write op
0812 06:37:12.174844 (+     4us) tablet.cc:685] Acquiring locks for 10 operations
0812 06:37:12.174908 (+    64us) tablet.cc:701] Row locks acquired
0812 06:37:12.174909 (+     1us) write_op.cc:260] PREPARE: finished
0812 06:37:12.174937 (+    28us) write_op.cc:270] Start()
0812 06:37:12.174954 (+    17us) write_op.cc:276] Timestamp: P: 1786516632174887 usec, L: 0
0812 06:37:12.174957 (+     3us) op_driver.cc:348] REPLICATION: starting
0812 06:37:12.175063 (+   106us) log.cc:844] Serialized 686 byte log entry
0812 06:37:16.163967 (+3988904us) op_driver.cc:464] REPLICATION: finished
0812 06:37:16.163973 (+     6us) op_driver.cc:389] HandleFailure(Aborted: Op aborted by new leader)
0812 06:37:16.163976 (+     3us) write_op.cc:489] Releasing partition, row and schema locks
0812 06:37:16.163999 (+    23us) write_op.cc:454] Released schema lock
0812 06:37:16.164005 (+     6us) write_op.cc:335] FINISH: Op aborted
Metrics: {"child_traces":[["op",{"num_ops":10,"prepare.queue_time_us":299,"prepare.run_cpu_time_us":454,"prepare.run_wall_time_us":713,"replication_time_us":3988976}]]}
I20260812 06:37:16.169840  9421 raft_consensus.cc:1217] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [term 23 FOLLOWER]: Deduplicated request from leader. Original: 22.3505->[22.3506-23.3515]   Dedup: 22.3509->[23.3510-23.3515]
I20260812 06:37:16.169948  9421 pending_rounds.cc:85] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790: Aborting all ops after (but not including) 3509
I20260812 06:37:16.169977  9421 pending_rounds.cc:107] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790: Aborting uncommitted WRITE_OP operation due to leader change: 22.3510
I20260812 06:37:16.827600  4804 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskCnZu03/build/debug/bin/kudu with pid 10556
W20260812 06:37:16.842768  7401 meta_cache.cc:302] tablet 7065bb403e3047dd9a0853c49318ef1f: replica 7962f587ccd842ad8a3f6ed9a5d7707e (127.4.177.3:40897) has failed: Network error: recv got EOF from 127.4.177.3:40897 (error 108)
W20260812 06:37:16.848150  6146 connection.cc:466] server connection from 127.4.177.3:33759 torn down before Call kudu.consensus.ConsensusService.UpdateConsensus from 127.4.177.3:33759 (request call id 36) could send its response
W20260812 06:37:16.856786  9313 connection.cc:466] server connection from 127.4.177.3:37635 torn down before Call kudu.consensus.ConsensusService.UpdateConsensus from 127.4.177.3:37635 (request call id 50) could send its response
W20260812 06:37:16.861989  7210 connection.cc:466] server connection from 127.4.177.3:41175 torn down before Call kudu.consensus.ConsensusService.UpdateConsensus from 127.4.177.3:41175 (request call id 22) could send its response
W20260812 06:37:16.912014  5945 connection.cc:466] server connection from 127.4.177.3:39797 torn down before Call kudu.consensus.ConsensusService.UpdateConsensus from 127.4.177.3:39797 (request call id 6) could send its response
I20260812 06:37:18.005653 11102 raft_consensus.cc:493] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 7962f587ccd842ad8a3f6ed9a5d7707e)
I20260812 06:37:18.005910 11103 raft_consensus.cc:493] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 7962f587ccd842ad8a3f6ed9a5d7707e)
I20260812 06:37:18.005880 11102 raft_consensus.cc:515] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df4308c852240859ada53d216ab69ae" member_type: VOTER last_known_addr { host: "127.4.177.6" port: 34269 } } peers { permanent_uuid: "42895a9bcba64aa484f6509605035d02" member_type: VOTER last_known_addr { host: "127.4.177.4" port: 38181 } } peers { permanent_uuid: "222b20de5a1145c6b8a11df8374e8790" member_type: VOTER last_known_addr { host: "127.4.177.5" port: 43521 } } peers { permanent_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 40897 } } peers { permanent_uuid: "8d90d61da62549208fe117b4384a469b" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 41539 } } peers { permanent_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" member_type: VOTER last_known_addr { host: "127.4.177.7" port: 46299 } } peers { permanent_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 41057 } }
I20260812 06:37:18.006045 11103 raft_consensus.cc:515] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df4308c852240859ada53d216ab69ae" member_type: VOTER last_known_addr { host: "127.4.177.6" port: 34269 } } peers { permanent_uuid: "42895a9bcba64aa484f6509605035d02" member_type: VOTER last_known_addr { host: "127.4.177.4" port: 38181 } } peers { permanent_uuid: "222b20de5a1145c6b8a11df8374e8790" member_type: VOTER last_known_addr { host: "127.4.177.5" port: 43521 } } peers { permanent_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 40897 } } peers { permanent_uuid: "8d90d61da62549208fe117b4384a469b" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 41539 } } peers { permanent_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" member_type: VOTER last_known_addr { host: "127.4.177.7" port: 46299 } } peers { permanent_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 41057 } }
I20260812 06:37:18.007040 11102 leader_election.cc:290] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 42895a9bcba64aa484f6509605035d02 (127.4.177.4:38181), 222b20de5a1145c6b8a11df8374e8790 (127.4.177.5:43521), 7962f587ccd842ad8a3f6ed9a5d7707e (127.4.177.3:40897), 8d90d61da62549208fe117b4384a469b (127.4.177.2:41539), b037d17b21f8422fa5e41b70eb4dcd08 (127.4.177.7:46299), ec6c8d818e744cdbb6b0d15b321010e0 (127.4.177.1:41057)
I20260812 06:37:18.007571  7322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "2df4308c852240859ada53d216ab69ae" candidate_term: 24 candidate_status { last_received { term: 23 index: 3627 } } ignore_live_leader: false dest_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" is_pre_election: true
I20260812 06:37:18.007668  9794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "42895a9bcba64aa484f6509605035d02" candidate_term: 24 candidate_status { last_received { term: 23 index: 3627 } } ignore_live_leader: false dest_uuid: "2df4308c852240859ada53d216ab69ae" is_pre_election: true
I20260812 06:37:18.007786  9794 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 42895a9bcba64aa484f6509605035d02 in term 23.
W20260812 06:37:18.007865  9714 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.4.177.3:40897: connect: Connection refused (error 111)
I20260812 06:37:18.008068  9425 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "42895a9bcba64aa484f6509605035d02" candidate_term: 24 candidate_status { last_received { term: 23 index: 3627 } } ignore_live_leader: false dest_uuid: "222b20de5a1145c6b8a11df8374e8790" is_pre_election: true
I20260812 06:37:18.008211  6238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "42895a9bcba64aa484f6509605035d02" candidate_term: 24 candidate_status { last_received { term: 23 index: 3627 } } ignore_live_leader: false dest_uuid: "8d90d61da62549208fe117b4384a469b" is_pre_election: true
I20260812 06:37:18.008301  6046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "42895a9bcba64aa484f6509605035d02" candidate_term: 24 candidate_status { last_received { term: 23 index: 3627 } } ignore_live_leader: false dest_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" is_pre_election: true
I20260812 06:37:18.008620  6048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "2df4308c852240859ada53d216ab69ae" candidate_term: 24 candidate_status { last_received { term: 23 index: 3627 } } ignore_live_leader: false dest_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" is_pre_election: true
I20260812 06:37:18.008762 11103 leader_election.cc:290] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 2df4308c852240859ada53d216ab69ae (127.4.177.6:34269), 222b20de5a1145c6b8a11df8374e8790 (127.4.177.5:43521), 7962f587ccd842ad8a3f6ed9a5d7707e (127.4.177.3:40897), 8d90d61da62549208fe117b4384a469b (127.4.177.2:41539), b037d17b21f8422fa5e41b70eb4dcd08 (127.4.177.7:46299), ec6c8d818e744cdbb6b0d15b321010e0 (127.4.177.1:41057)
I20260812 06:37:18.008819  6700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "2df4308c852240859ada53d216ab69ae" candidate_term: 24 candidate_status { last_received { term: 23 index: 3627 } } ignore_live_leader: false dest_uuid: "42895a9bcba64aa484f6509605035d02" is_pre_election: true
I20260812 06:37:18.008967  6700 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2df4308c852240859ada53d216ab69ae in term 23.
I20260812 06:37:18.009102  6238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "2df4308c852240859ada53d216ab69ae" candidate_term: 24 candidate_status { last_received { term: 23 index: 3627 } } ignore_live_leader: false dest_uuid: "8d90d61da62549208fe117b4384a469b" is_pre_election: true
I20260812 06:37:18.009264  9421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "2df4308c852240859ada53d216ab69ae" candidate_term: 24 candidate_status { last_received { term: 23 index: 3627 } } ignore_live_leader: false dest_uuid: "222b20de5a1145c6b8a11df8374e8790" is_pre_election: true
I20260812 06:37:18.009620  7322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "42895a9bcba64aa484f6509605035d02" candidate_term: 24 candidate_status { last_received { term: 23 index: 3627 } } ignore_live_leader: false dest_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" is_pre_election: true
I20260812 06:37:18.009711  9714 leader_election.cc:304] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 6 responses out of 7 voters: 2 yes votes; 4 no votes. yes voters: 2df4308c852240859ada53d216ab69ae, 42895a9bcba64aa484f6509605035d02; no voters: 222b20de5a1145c6b8a11df8374e8790, 8d90d61da62549208fe117b4384a469b, b037d17b21f8422fa5e41b70eb4dcd08, ec6c8d818e744cdbb6b0d15b321010e0
I20260812 06:37:18.009996 11102 raft_consensus.cc:2749] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260812 06:37:18.010071  6595 leader_election.cc:304] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 6 responses out of 7 voters: 2 yes votes; 4 no votes. yes voters: 2df4308c852240859ada53d216ab69ae, 42895a9bcba64aa484f6509605035d02; no voters: 222b20de5a1145c6b8a11df8374e8790, 8d90d61da62549208fe117b4384a469b, b037d17b21f8422fa5e41b70eb4dcd08, ec6c8d818e744cdbb6b0d15b321010e0
W20260812 06:37:18.010298  9714 leader_election.cc:336] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 7962f587ccd842ad8a3f6ed9a5d7707e (127.4.177.3:40897): Network error: Client connection negotiation failed: client connection to 127.4.177.3:40897: connect: Connection refused (error 111)
I20260812 06:37:18.010373 11103 raft_consensus.cc:2749] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260812 06:37:18.010679  6597 leader_election.cc:336] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 7962f587ccd842ad8a3f6ed9a5d7707e (127.4.177.3:40897): Network error: Client connection negotiation failed: client connection to 127.4.177.3:40897: connect: Connection refused (error 111)
I20260812 06:37:18.346057 11112 raft_consensus.cc:493] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 7962f587ccd842ad8a3f6ed9a5d7707e)
I20260812 06:37:18.346288 11112 raft_consensus.cc:515] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df4308c852240859ada53d216ab69ae" member_type: VOTER last_known_addr { host: "127.4.177.6" port: 34269 } } peers { permanent_uuid: "42895a9bcba64aa484f6509605035d02" member_type: VOTER last_known_addr { host: "127.4.177.4" port: 38181 } } peers { permanent_uuid: "222b20de5a1145c6b8a11df8374e8790" member_type: VOTER last_known_addr { host: "127.4.177.5" port: 43521 } } peers { permanent_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 40897 } } peers { permanent_uuid: "8d90d61da62549208fe117b4384a469b" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 41539 } } peers { permanent_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" member_type: VOTER last_known_addr { host: "127.4.177.7" port: 46299 } } peers { permanent_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 41057 } }
I20260812 06:37:18.347136 11112 leader_election.cc:290] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 2df4308c852240859ada53d216ab69ae (127.4.177.6:34269), 42895a9bcba64aa484f6509605035d02 (127.4.177.4:38181), 222b20de5a1145c6b8a11df8374e8790 (127.4.177.5:43521), 7962f587ccd842ad8a3f6ed9a5d7707e (127.4.177.3:40897), 8d90d61da62549208fe117b4384a469b (127.4.177.2:41539), ec6c8d818e744cdbb6b0d15b321010e0 (127.4.177.1:41057)
I20260812 06:37:18.347707  6700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" candidate_term: 24 candidate_status { last_received { term: 23 index: 3541 } } ignore_live_leader: false dest_uuid: "42895a9bcba64aa484f6509605035d02" is_pre_election: true
I20260812 06:37:18.347738  9794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" candidate_term: 24 candidate_status { last_received { term: 23 index: 3541 } } ignore_live_leader: false dest_uuid: "2df4308c852240859ada53d216ab69ae" is_pre_election: true
I20260812 06:37:18.347821  9421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" candidate_term: 24 candidate_status { last_received { term: 23 index: 3541 } } ignore_live_leader: false dest_uuid: "222b20de5a1145c6b8a11df8374e8790" is_pre_election: true
I20260812 06:37:18.347913  6700 raft_consensus.cc:2410] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b037d17b21f8422fa5e41b70eb4dcd08 for term 24 because replica has last-logged OpId of term: 23 index: 3627, which is greater than that of the candidate, which has last-logged OpId of term: 23 index: 3541.
I20260812 06:37:18.347945  9794 raft_consensus.cc:2410] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b037d17b21f8422fa5e41b70eb4dcd08 for term 24 because replica has last-logged OpId of term: 23 index: 3627, which is greater than that of the candidate, which has last-logged OpId of term: 23 index: 3541.
I20260812 06:37:18.348423  6238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" candidate_term: 24 candidate_status { last_received { term: 23 index: 3541 } } ignore_live_leader: false dest_uuid: "8d90d61da62549208fe117b4384a469b" is_pre_election: true
I20260812 06:37:18.348615  6238 raft_consensus.cc:2410] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b037d17b21f8422fa5e41b70eb4dcd08 for term 24 because replica has last-logged OpId of term: 23 index: 3576, which is greater than that of the candidate, which has last-logged OpId of term: 23 index: 3541.
I20260812 06:37:18.348676  6048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" candidate_term: 24 candidate_status { last_received { term: 23 index: 3541 } } ignore_live_leader: false dest_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" is_pre_election: true
I20260812 06:37:18.348807  6048 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P ec6c8d818e744cdbb6b0d15b321010e0 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b037d17b21f8422fa5e41b70eb4dcd08 in term 23.
I20260812 06:37:18.348821 11116 raft_consensus.cc:493] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 7962f587ccd842ad8a3f6ed9a5d7707e)
I20260812 06:37:18.348954 11116 raft_consensus.cc:515] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df4308c852240859ada53d216ab69ae" member_type: VOTER last_known_addr { host: "127.4.177.6" port: 34269 } } peers { permanent_uuid: "42895a9bcba64aa484f6509605035d02" member_type: VOTER last_known_addr { host: "127.4.177.4" port: 38181 } } peers { permanent_uuid: "222b20de5a1145c6b8a11df8374e8790" member_type: VOTER last_known_addr { host: "127.4.177.5" port: 43521 } } peers { permanent_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 40897 } } peers { permanent_uuid: "8d90d61da62549208fe117b4384a469b" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 41539 } } peers { permanent_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" member_type: VOTER last_known_addr { host: "127.4.177.7" port: 46299 } } peers { permanent_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 41057 } }
I20260812 06:37:18.349046 11115 raft_consensus.cc:493] T 7065bb403e3047dd9a0853c49318ef1f P ec6c8d818e744cdbb6b0d15b321010e0 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 7962f587ccd842ad8a3f6ed9a5d7707e)
I20260812 06:37:18.349107  7211 leader_election.cc:304] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 5 responses out of 7 voters: 1 yes votes; 4 no votes. yes voters: b037d17b21f8422fa5e41b70eb4dcd08; no voters: 222b20de5a1145c6b8a11df8374e8790, 2df4308c852240859ada53d216ab69ae, 42895a9bcba64aa484f6509605035d02, 8d90d61da62549208fe117b4384a469b
I20260812 06:37:18.349176 11115 raft_consensus.cc:515] T 7065bb403e3047dd9a0853c49318ef1f P ec6c8d818e744cdbb6b0d15b321010e0 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df4308c852240859ada53d216ab69ae" member_type: VOTER last_known_addr { host: "127.4.177.6" port: 34269 } } peers { permanent_uuid: "42895a9bcba64aa484f6509605035d02" member_type: VOTER last_known_addr { host: "127.4.177.4" port: 38181 } } peers { permanent_uuid: "222b20de5a1145c6b8a11df8374e8790" member_type: VOTER last_known_addr { host: "127.4.177.5" port: 43521 } } peers { permanent_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 40897 } } peers { permanent_uuid: "8d90d61da62549208fe117b4384a469b" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 41539 } } peers { permanent_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" member_type: VOTER last_known_addr { host: "127.4.177.7" port: 46299 } } peers { permanent_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 41057 } }
I20260812 06:37:18.349445 11112 raft_consensus.cc:2749] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260812 06:37:18.350003  7211 leader_election.cc:336] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 7962f587ccd842ad8a3f6ed9a5d7707e (127.4.177.3:40897): Network error: Client connection negotiation failed: client connection to 127.4.177.3:40897: connect: Connection refused (error 111)
I20260812 06:37:18.350041  9794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" candidate_term: 24 candidate_status { last_received { term: 23 index: 3514 } } ignore_live_leader: false dest_uuid: "2df4308c852240859ada53d216ab69ae" is_pre_election: true
I20260812 06:37:18.350198  9794 raft_consensus.cc:2410] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ec6c8d818e744cdbb6b0d15b321010e0 for term 24 because replica has last-logged OpId of term: 23 index: 3627, which is greater than that of the candidate, which has last-logged OpId of term: 23 index: 3514.
I20260812 06:37:18.350337  9799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "8d90d61da62549208fe117b4384a469b" candidate_term: 24 candidate_status { last_received { term: 23 index: 3576 } } ignore_live_leader: false dest_uuid: "2df4308c852240859ada53d216ab69ae" is_pre_election: true
I20260812 06:37:18.350493  9799 raft_consensus.cc:2410] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8d90d61da62549208fe117b4384a469b for term 24 because replica has last-logged OpId of term: 23 index: 3627, which is greater than that of the candidate, which has last-logged OpId of term: 23 index: 3576.
I20260812 06:37:18.350720 11115 leader_election.cc:290] T 7065bb403e3047dd9a0853c49318ef1f P ec6c8d818e744cdbb6b0d15b321010e0 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 2df4308c852240859ada53d216ab69ae (127.4.177.6:34269), 42895a9bcba64aa484f6509605035d02 (127.4.177.4:38181), 222b20de5a1145c6b8a11df8374e8790 (127.4.177.5:43521), 7962f587ccd842ad8a3f6ed9a5d7707e (127.4.177.3:40897), 8d90d61da62549208fe117b4384a469b (127.4.177.2:41539), b037d17b21f8422fa5e41b70eb4dcd08 (127.4.177.7:46299)
I20260812 06:37:18.350939  6700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" candidate_term: 24 candidate_status { last_received { term: 23 index: 3514 } } ignore_live_leader: false dest_uuid: "42895a9bcba64aa484f6509605035d02" is_pre_election: true
I20260812 06:37:18.351151  6700 raft_consensus.cc:2410] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ec6c8d818e744cdbb6b0d15b321010e0 for term 24 because replica has last-logged OpId of term: 23 index: 3627, which is greater than that of the candidate, which has last-logged OpId of term: 23 index: 3514.
I20260812 06:37:18.351159  6701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "8d90d61da62549208fe117b4384a469b" candidate_term: 24 candidate_status { last_received { term: 23 index: 3576 } } ignore_live_leader: false dest_uuid: "42895a9bcba64aa484f6509605035d02" is_pre_election: true
I20260812 06:37:18.351398  6701 raft_consensus.cc:2410] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8d90d61da62549208fe117b4384a469b for term 24 because replica has last-logged OpId of term: 23 index: 3627, which is greater than that of the candidate, which has last-logged OpId of term: 23 index: 3576.
I20260812 06:37:18.351452  9421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" candidate_term: 24 candidate_status { last_received { term: 23 index: 3514 } } ignore_live_leader: false dest_uuid: "222b20de5a1145c6b8a11df8374e8790" is_pre_election: true
I20260812 06:37:18.351538  7322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "8d90d61da62549208fe117b4384a469b" candidate_term: 24 candidate_status { last_received { term: 23 index: 3576 } } ignore_live_leader: false dest_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" is_pre_election: true
I20260812 06:37:18.351675  7322 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8d90d61da62549208fe117b4384a469b in term 23.
I20260812 06:37:18.351884  7320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" candidate_term: 24 candidate_status { last_received { term: 23 index: 3514 } } ignore_live_leader: false dest_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" is_pre_election: true
I20260812 06:37:18.352026  7320 raft_consensus.cc:2410] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ec6c8d818e744cdbb6b0d15b321010e0 for term 24 because replica has last-logged OpId of term: 23 index: 3541, which is greater than that of the candidate, which has last-logged OpId of term: 23 index: 3514.
I20260812 06:37:18.352300  9421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "8d90d61da62549208fe117b4384a469b" candidate_term: 24 candidate_status { last_received { term: 23 index: 3576 } } ignore_live_leader: false dest_uuid: "222b20de5a1145c6b8a11df8374e8790" is_pre_election: true
I20260812 06:37:18.352182 11116 leader_election.cc:290] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 2df4308c852240859ada53d216ab69ae (127.4.177.6:34269), 42895a9bcba64aa484f6509605035d02 (127.4.177.4:38181), 222b20de5a1145c6b8a11df8374e8790 (127.4.177.5:43521), 7962f587ccd842ad8a3f6ed9a5d7707e (127.4.177.3:40897), b037d17b21f8422fa5e41b70eb4dcd08 (127.4.177.7:46299), ec6c8d818e744cdbb6b0d15b321010e0 (127.4.177.1:41057)
I20260812 06:37:18.352535  5945 leader_election.cc:304] T 7065bb403e3047dd9a0853c49318ef1f P ec6c8d818e744cdbb6b0d15b321010e0 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 5 responses out of 7 voters: 1 yes votes; 4 no votes. yes voters: ec6c8d818e744cdbb6b0d15b321010e0; no voters: 222b20de5a1145c6b8a11df8374e8790, 2df4308c852240859ada53d216ab69ae, 42895a9bcba64aa484f6509605035d02, b037d17b21f8422fa5e41b70eb4dcd08
I20260812 06:37:18.352505  6238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" candidate_term: 24 candidate_status { last_received { term: 23 index: 3514 } } ignore_live_leader: false dest_uuid: "8d90d61da62549208fe117b4384a469b" is_pre_election: true
I20260812 06:37:18.352609  6048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "8d90d61da62549208fe117b4384a469b" candidate_term: 24 candidate_status { last_received { term: 23 index: 3576 } } ignore_live_leader: false dest_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" is_pre_election: true
I20260812 06:37:18.352691  6238 raft_consensus.cc:2410] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ec6c8d818e744cdbb6b0d15b321010e0 for term 24 because replica has last-logged OpId of term: 23 index: 3576, which is greater than that of the candidate, which has last-logged OpId of term: 23 index: 3514.
I20260812 06:37:18.352737 11115 raft_consensus.cc:2749] T 7065bb403e3047dd9a0853c49318ef1f P ec6c8d818e744cdbb6b0d15b321010e0 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260812 06:37:18.352939  6048 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P ec6c8d818e744cdbb6b0d15b321010e0 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8d90d61da62549208fe117b4384a469b in term 23.
W20260812 06:37:18.354020  5945 leader_election.cc:336] T 7065bb403e3047dd9a0853c49318ef1f P ec6c8d818e744cdbb6b0d15b321010e0 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 7962f587ccd842ad8a3f6ed9a5d7707e (127.4.177.3:40897): Network error: Client connection negotiation failed: client connection to 127.4.177.3:40897: connect: Connection refused (error 111)
W20260812 06:37:18.354048  6150 leader_election.cc:336] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 7962f587ccd842ad8a3f6ed9a5d7707e (127.4.177.3:40897): Network error: Client connection negotiation failed: client connection to 127.4.177.3:40897: connect: Connection refused (error 111)
I20260812 06:37:18.354146  6150 leader_election.cc:304] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 7 responses out of 7 voters: 3 yes votes; 4 no votes. yes voters: 8d90d61da62549208fe117b4384a469b, b037d17b21f8422fa5e41b70eb4dcd08, ec6c8d818e744cdbb6b0d15b321010e0; no voters: 222b20de5a1145c6b8a11df8374e8790, 2df4308c852240859ada53d216ab69ae, 42895a9bcba64aa484f6509605035d02, 7962f587ccd842ad8a3f6ed9a5d7707e
I20260812 06:37:18.354352 11116 raft_consensus.cc:2749] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260812 06:37:18.356608 11131 raft_consensus.cc:493] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 7962f587ccd842ad8a3f6ed9a5d7707e)
I20260812 06:37:18.356706 11131 raft_consensus.cc:515] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df4308c852240859ada53d216ab69ae" member_type: VOTER last_known_addr { host: "127.4.177.6" port: 34269 } } peers { permanent_uuid: "42895a9bcba64aa484f6509605035d02" member_type: VOTER last_known_addr { host: "127.4.177.4" port: 38181 } } peers { permanent_uuid: "222b20de5a1145c6b8a11df8374e8790" member_type: VOTER last_known_addr { host: "127.4.177.5" port: 43521 } } peers { permanent_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 40897 } } peers { permanent_uuid: "8d90d61da62549208fe117b4384a469b" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 41539 } } peers { permanent_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" member_type: VOTER last_known_addr { host: "127.4.177.7" port: 46299 } } peers { permanent_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 41057 } }
I20260812 06:37:18.357383 11131 leader_election.cc:290] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 2df4308c852240859ada53d216ab69ae (127.4.177.6:34269), 42895a9bcba64aa484f6509605035d02 (127.4.177.4:38181), 7962f587ccd842ad8a3f6ed9a5d7707e (127.4.177.3:40897), 8d90d61da62549208fe117b4384a469b (127.4.177.2:41539), b037d17b21f8422fa5e41b70eb4dcd08 (127.4.177.7:46299), ec6c8d818e744cdbb6b0d15b321010e0 (127.4.177.1:41057)
I20260812 06:37:18.357724  9799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "222b20de5a1145c6b8a11df8374e8790" candidate_term: 24 candidate_status { last_received { term: 23 index: 3619 } } ignore_live_leader: false dest_uuid: "2df4308c852240859ada53d216ab69ae" is_pre_election: true
I20260812 06:37:18.357770  7320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "222b20de5a1145c6b8a11df8374e8790" candidate_term: 24 candidate_status { last_received { term: 23 index: 3619 } } ignore_live_leader: false dest_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" is_pre_election: true
I20260812 06:37:18.357903  9799 raft_consensus.cc:2410] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 222b20de5a1145c6b8a11df8374e8790 for term 24 because replica has last-logged OpId of term: 23 index: 3627, which is greater than that of the candidate, which has last-logged OpId of term: 23 index: 3619.
I20260812 06:37:18.357903  7320 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 222b20de5a1145c6b8a11df8374e8790 in term 23.
I20260812 06:37:18.358297  6048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "222b20de5a1145c6b8a11df8374e8790" candidate_term: 24 candidate_status { last_received { term: 23 index: 3619 } } ignore_live_leader: false dest_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" is_pre_election: true
I20260812 06:37:18.358166  6701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "222b20de5a1145c6b8a11df8374e8790" candidate_term: 24 candidate_status { last_received { term: 23 index: 3619 } } ignore_live_leader: false dest_uuid: "42895a9bcba64aa484f6509605035d02" is_pre_election: true
I20260812 06:37:18.358430  6048 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P ec6c8d818e744cdbb6b0d15b321010e0 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 222b20de5a1145c6b8a11df8374e8790 in term 23.
I20260812 06:37:18.358523  6701 raft_consensus.cc:2410] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 222b20de5a1145c6b8a11df8374e8790 for term 24 because replica has last-logged OpId of term: 23 index: 3627, which is greater than that of the candidate, which has last-logged OpId of term: 23 index: 3619.
I20260812 06:37:18.358536  6238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "222b20de5a1145c6b8a11df8374e8790" candidate_term: 24 candidate_status { last_received { term: 23 index: 3619 } } ignore_live_leader: false dest_uuid: "8d90d61da62549208fe117b4384a469b" is_pre_election: true
I20260812 06:37:18.358649  6238 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 222b20de5a1145c6b8a11df8374e8790 in term 23.
I20260812 06:37:18.359127  9315 leader_election.cc:304] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 6 responses out of 7 voters: 4 yes votes; 2 no votes. yes voters: 222b20de5a1145c6b8a11df8374e8790, 8d90d61da62549208fe117b4384a469b, b037d17b21f8422fa5e41b70eb4dcd08, ec6c8d818e744cdbb6b0d15b321010e0; no voters: 2df4308c852240859ada53d216ab69ae, 42895a9bcba64aa484f6509605035d02
W20260812 06:37:18.359293  9315 leader_election.cc:336] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 7962f587ccd842ad8a3f6ed9a5d7707e (127.4.177.3:40897): Network error: Client connection negotiation failed: client connection to 127.4.177.3:40897: connect: Connection refused (error 111)
I20260812 06:37:18.359304 11131 raft_consensus.cc:2804] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260812 06:37:18.359406 11131 raft_consensus.cc:493] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [term 23 FOLLOWER]: Starting leader election (detected failure of leader 7962f587ccd842ad8a3f6ed9a5d7707e)
I20260812 06:37:18.359457 11131 raft_consensus.cc:3060] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [term 23 FOLLOWER]: Advancing to term 24
I20260812 06:37:18.360327 11131 raft_consensus.cc:515] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df4308c852240859ada53d216ab69ae" member_type: VOTER last_known_addr { host: "127.4.177.6" port: 34269 } } peers { permanent_uuid: "42895a9bcba64aa484f6509605035d02" member_type: VOTER last_known_addr { host: "127.4.177.4" port: 38181 } } peers { permanent_uuid: "222b20de5a1145c6b8a11df8374e8790" member_type: VOTER last_known_addr { host: "127.4.177.5" port: 43521 } } peers { permanent_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 40897 } } peers { permanent_uuid: "8d90d61da62549208fe117b4384a469b" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 41539 } } peers { permanent_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" member_type: VOTER last_known_addr { host: "127.4.177.7" port: 46299 } } peers { permanent_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 41057 } }
I20260812 06:37:18.360939 11131 leader_election.cc:290] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [CANDIDATE]: Term 24 election: Requested vote from peers 2df4308c852240859ada53d216ab69ae (127.4.177.6:34269), 42895a9bcba64aa484f6509605035d02 (127.4.177.4:38181), 7962f587ccd842ad8a3f6ed9a5d7707e (127.4.177.3:40897), 8d90d61da62549208fe117b4384a469b (127.4.177.2:41539), b037d17b21f8422fa5e41b70eb4dcd08 (127.4.177.7:46299), ec6c8d818e744cdbb6b0d15b321010e0 (127.4.177.1:41057)
I20260812 06:37:18.361093  6701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "222b20de5a1145c6b8a11df8374e8790" candidate_term: 24 candidate_status { last_received { term: 23 index: 3619 } } ignore_live_leader: false dest_uuid: "42895a9bcba64aa484f6509605035d02"
I20260812 06:37:18.361093  9799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "222b20de5a1145c6b8a11df8374e8790" candidate_term: 24 candidate_status { last_received { term: 23 index: 3619 } } ignore_live_leader: false dest_uuid: "2df4308c852240859ada53d216ab69ae"
I20260812 06:37:18.361207  9799 raft_consensus.cc:3060] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [term 23 FOLLOWER]: Advancing to term 24
I20260812 06:37:18.361207  6701 raft_consensus.cc:3060] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [term 23 FOLLOWER]: Advancing to term 24
I20260812 06:37:18.361433  6238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "222b20de5a1145c6b8a11df8374e8790" candidate_term: 24 candidate_status { last_received { term: 23 index: 3619 } } ignore_live_leader: false dest_uuid: "8d90d61da62549208fe117b4384a469b"
I20260812 06:37:18.361434  7320 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "222b20de5a1145c6b8a11df8374e8790" candidate_term: 24 candidate_status { last_received { term: 23 index: 3619 } } ignore_live_leader: false dest_uuid: "b037d17b21f8422fa5e41b70eb4dcd08"
I20260812 06:37:18.361541  7320 raft_consensus.cc:3060] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08 [term 23 FOLLOWER]: Advancing to term 24
I20260812 06:37:18.361541  6238 raft_consensus.cc:3060] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b [term 23 FOLLOWER]: Advancing to term 24
I20260812 06:37:18.362368  9799 raft_consensus.cc:2410] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 222b20de5a1145c6b8a11df8374e8790 for term 24 because replica has last-logged OpId of term: 23 index: 3627, which is greater than that of the candidate, which has last-logged OpId of term: 23 index: 3619.
I20260812 06:37:18.362368  6701 raft_consensus.cc:2410] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 222b20de5a1145c6b8a11df8374e8790 for term 24 because replica has last-logged OpId of term: 23 index: 3627, which is greater than that of the candidate, which has last-logged OpId of term: 23 index: 3619.
I20260812 06:37:18.362586  7320 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 222b20de5a1145c6b8a11df8374e8790 in term 24.
I20260812 06:37:18.362586  6238 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 222b20de5a1145c6b8a11df8374e8790 in term 24.
I20260812 06:37:18.363008  6048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "222b20de5a1145c6b8a11df8374e8790" candidate_term: 24 candidate_status { last_received { term: 23 index: 3619 } } ignore_live_leader: false dest_uuid: "ec6c8d818e744cdbb6b0d15b321010e0"
I20260812 06:37:18.363114  6048 raft_consensus.cc:3060] T 7065bb403e3047dd9a0853c49318ef1f P ec6c8d818e744cdbb6b0d15b321010e0 [term 23 FOLLOWER]: Advancing to term 24
I20260812 06:37:18.363873  6048 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P ec6c8d818e744cdbb6b0d15b321010e0 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 222b20de5a1145c6b8a11df8374e8790 in term 24.
I20260812 06:37:18.364200  9315 leader_election.cc:304] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 6 responses out of 7 voters: 4 yes votes; 2 no votes. yes voters: 222b20de5a1145c6b8a11df8374e8790, 8d90d61da62549208fe117b4384a469b, b037d17b21f8422fa5e41b70eb4dcd08, ec6c8d818e744cdbb6b0d15b321010e0; no voters: 2df4308c852240859ada53d216ab69ae, 42895a9bcba64aa484f6509605035d02
I20260812 06:37:18.364377 11131 raft_consensus.cc:2804] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [term 24 FOLLOWER]: Leader election won for term 24
I20260812 06:37:18.364576 11131 raft_consensus.cc:697] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [term 24 LEADER]: Becoming Leader. State: Replica: 222b20de5a1145c6b8a11df8374e8790, State: Running, Role: LEADER
I20260812 06:37:18.364976 11131 consensus_queue.cc:237] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3617, Committed index: 3617, Last appended: 23.3619, Last appended by leader: 3619, Current term: 24, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df4308c852240859ada53d216ab69ae" member_type: VOTER last_known_addr { host: "127.4.177.6" port: 34269 } } peers { permanent_uuid: "42895a9bcba64aa484f6509605035d02" member_type: VOTER last_known_addr { host: "127.4.177.4" port: 38181 } } peers { permanent_uuid: "222b20de5a1145c6b8a11df8374e8790" member_type: VOTER last_known_addr { host: "127.4.177.5" port: 43521 } } peers { permanent_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 40897 } } peers { permanent_uuid: "8d90d61da62549208fe117b4384a469b" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 41539 } } peers { permanent_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" member_type: VOTER last_known_addr { host: "127.4.177.7" port: 46299 } } peers { permanent_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 41057 } }
W20260812 06:37:18.365094  9315 leader_election.cc:336] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [CANDIDATE]: Term 24 election: RPC error from VoteRequest() call to peer 7962f587ccd842ad8a3f6ed9a5d7707e (127.4.177.3:40897): Network error: Client connection negotiation failed: client connection to 127.4.177.3:40897: connect: Connection refused (error 111)
I20260812 06:37:18.367871  5808 catalog_manager.cc:5719] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 reported cstate change: term changed from 23 to 24, leader changed from 7962f587ccd842ad8a3f6ed9a5d7707e (127.4.177.3) to 222b20de5a1145c6b8a11df8374e8790 (127.4.177.5). New cstate: current_term: 24 leader_uuid: "222b20de5a1145c6b8a11df8374e8790" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df4308c852240859ada53d216ab69ae" member_type: VOTER last_known_addr { host: "127.4.177.6" port: 34269 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "42895a9bcba64aa484f6509605035d02" member_type: VOTER last_known_addr { host: "127.4.177.4" port: 38181 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "222b20de5a1145c6b8a11df8374e8790" member_type: VOTER last_known_addr { host: "127.4.177.5" port: 43521 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 40897 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8d90d61da62549208fe117b4384a469b" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 41539 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" member_type: VOTER last_known_addr { host: "127.4.177.7" port: 46299 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 41057 } health_report { overall_health: UNKNOWN } } }
I20260812 06:37:18.448118  4804 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskCnZu03/build/debug/bin/kudu
/tmp/dist-test-taskCnZu03/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.177.3:40897
--local_ip_for_outbound_sockets=127.4.177.3
--tserver_master_addrs=127.4.177.62:42183
--webserver_port=44527
--webserver_interface=127.4.177.3
--builtin_ntp_servers=127.4.177.20:34335
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--log_cache_size_limit_mb=10
--consensus_rpc_timeout_ms=1000 with env {}
W20260812 06:37:18.561430 11142 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:37:18.561663 11142 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:37:18.561708 11142 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 06:37:18.565176 11142 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:37:18.565320 11142 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.177.3
I20260812 06:37:18.569473 11142 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-2/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.177.20:34335
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--consensus_rpc_timeout_ms=1000
--log_cache_size_limit_mb=10
--fs_data_dirs=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.177.3:40897
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--webserver_interface=127.4.177.3
--webserver_port=44527
--tserver_master_addrs=127.4.177.62:42183
--never_fsync=true
--heap_profile_path=/tmp/kudu.11142
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.177.3

Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 06:37:18.570643 11142 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:37:18.571795 11142 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 06:37:18.578774 11152 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:37:18.578799 11151 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:37:18.578893 11142 server_base.cc:1061] running on GCE node
W20260812 06:37:18.578773 11156 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:37:18.579388 11142 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:37:18.579978 11142 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:37:18.581163 11142 hybrid_clock.cc:648] HybridClock initialized: now 1786516638581152 us; error 32 us; skew 500 ppm
I20260812 06:37:18.582942 11142 webserver.cc:533] Webserver started at http://127.4.177.3:44527/ using document root <none> and password file <none>
I20260812 06:37:18.583501 11142 fs_manager.cc:362] Metadata directory not provided
I20260812 06:37:18.583593 11142 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:37:18.586901 11142 fs_manager.cc:714] Time spent opening directory manager: real 0.002s	user 0.003s	sys 0.000s
I20260812 06:37:18.588814 11167 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 06:37:18.589829 11142 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260812 06:37:18.589954 11142 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-2/data,/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-2/wal
uuid: "7962f587ccd842ad8a3f6ed9a5d7707e"
format_stamp: "Formatted at 2026-08-12 06:36:11 on dist-test-slave-jztv"
I20260812 06:37:18.590382 11142 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-2/wal
metadata directory: /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-2/wal
1 data directories: /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:37:18.615593 11142 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 06:37:18.616340 11142 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:37:18.616542 11142 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:37:18.617153 11142 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260812 06:37:18.618466 11178 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
I20260812 06:37:18.621846 11142 ts_tablet_manager.cc:585] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260812 06:37:18.621922 11142 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.004s	user 0.000s	sys 0.000s
I20260812 06:37:18.622013 11142 ts_tablet_manager.cc:600] Registering tablets (0/1 complete)
I20260812 06:37:18.623502 11142 ts_tablet_manager.cc:616] Registered 1 tablets
I20260812 06:37:18.623577 11142 ts_tablet_manager.cc:595] Time spent register tablets: real 0.002s	user 0.000s	sys 0.000s
I20260812 06:37:18.623874 11178 tablet_bootstrap.cc:492] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e: Bootstrap starting.
I20260812 06:37:18.635468 11142 rpc_server.cc:307] RPC server started. Bound to: 127.4.177.3:40897
I20260812 06:37:18.635493 11390 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.4.177.3:40897 every 8 connection(s)
I20260812 06:37:18.636482 11142 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-2/data/info.pb
I20260812 06:37:18.637079  4804 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskCnZu03/build/debug/bin/kudu as pid 11142
I20260812 06:37:18.649744 11392 heartbeater.cc:344] Connected to a master server at 127.4.177.62:42183
I20260812 06:37:18.649994 11392 heartbeater.cc:461] Registering TS with master...
I20260812 06:37:18.650563 11392 heartbeater.cc:507] Master 127.4.177.62:42183 requested a full tablet report, sending...
I20260812 06:37:18.652280  5808 ts_manager.cc:194] Re-registered known tserver with Master: 7962f587ccd842ad8a3f6ed9a5d7707e (127.4.177.3:40897)
I20260812 06:37:18.653277  5808 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.4.177.3:35937
W20260812 06:37:18.784691  9315 consensus_peers.cc:597] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 -> Peer 7962f587ccd842ad8a3f6ed9a5d7707e (127.4.177.3:40897): Couldn't send request to peer 7962f587ccd842ad8a3f6ed9a5d7707e. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260812 06:37:18.787616  7320 raft_consensus.cc:1275] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08 [term 24 FOLLOWER]: Refusing update from remote peer 222b20de5a1145c6b8a11df8374e8790: Log matching property violated. Preceding OpId in replica: term: 23 index: 3541. Preceding OpId from leader: term: 24 index: 3620. (index mismatch)
I20260812 06:37:18.788264 11131 consensus_queue.cc:1048] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" member_type: VOTER last_known_addr { host: "127.4.177.7" port: 46299 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3620, Last known committed idx: 3539, Time since last communication: 0.000s
I20260812 06:37:18.861470 11178 log.cc:826] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e: Log is configured to *not* fsync() on all Append() calls
I20260812 06:37:18.931530  9799 raft_consensus.cc:1275] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [term 24 FOLLOWER]: Refusing update from remote peer 222b20de5a1145c6b8a11df8374e8790: Log matching property violated. Preceding OpId in replica: term: 23 index: 3627. Preceding OpId from leader: term: 24 index: 3620. (term mismatch)
I20260812 06:37:18.931649  9799 pending_rounds.cc:85] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae: Aborting all ops after (but not including) 3619
I20260812 06:37:18.931677  9799 pending_rounds.cc:107] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae: Aborting uncommitted WRITE_OP operation due to leader change: 23.3620
I20260812 06:37:18.931982  9799 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:41528 (ReqId={client: 3067f3aef2a44cc4bb7f2ccd6acf83db, seq_no=3601, attempt_no=1}) took 2065 ms. Trace:
I20260812 06:37:18.932081  9799 rpcz_store.cc:276] 0812 06:37:16.866417 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 06:37:16.866459 (+    42us) service_pool.cc:224] Handling call
0812 06:37:18.931969 (+2065510us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260812 06:37:18.932407  9799 pending_rounds.cc:107] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae: Aborting uncommitted WRITE_OP operation due to leader change: 23.3621
I20260812 06:37:18.932613  9799 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:41528 (ReqId={client: 3067f3aef2a44cc4bb7f2ccd6acf83db, seq_no=3602, attempt_no=1}) took 2066 ms. Trace:
I20260812 06:37:18.932680  9799 rpcz_store.cc:276] 0812 06:37:16.866030 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 06:37:16.866139 (+   109us) service_pool.cc:224] Handling call
0812 06:37:18.932604 (+2066465us) inbound_call.cc:177] Queueing failure response
Metrics: {"spinlock_wait_cycles":16640}
I20260812 06:37:18.932962  9799 pending_rounds.cc:107] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae: Aborting uncommitted WRITE_OP operation due to leader change: 23.3622
I20260812 06:37:18.933135  9799 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:41528 (ReqId={client: 3067f3aef2a44cc4bb7f2ccd6acf83db, seq_no=3604, attempt_no=1}) took 2066 ms. Trace:
I20260812 06:37:18.933171  9799 rpcz_store.cc:276] 0812 06:37:16.866314 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 06:37:16.866357 (+    43us) service_pool.cc:224] Handling call
0812 06:37:18.933127 (+2066770us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260812 06:37:18.933334  9799 pending_rounds.cc:107] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae: Aborting uncommitted WRITE_OP operation due to leader change: 23.3623
I20260812 06:37:18.933483  9799 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:41528 (ReqId={client: 3067f3aef2a44cc4bb7f2ccd6acf83db, seq_no=3603, attempt_no=1}) took 2067 ms. Trace:
I20260812 06:37:18.933517  9799 rpcz_store.cc:276] 0812 06:37:16.866178 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 06:37:16.866253 (+    75us) service_pool.cc:224] Handling call
0812 06:37:18.933476 (+2067223us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260812 06:37:18.933676  9799 pending_rounds.cc:107] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae: Aborting uncommitted WRITE_OP operation due to leader change: 23.3624
I20260812 06:37:18.933818  9799 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:41528 (ReqId={client: 3067f3aef2a44cc4bb7f2ccd6acf83db, seq_no=3605, attempt_no=1}) took 2067 ms. Trace:
I20260812 06:37:18.933851  9799 rpcz_store.cc:276] 0812 06:37:16.866262 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 06:37:16.866307 (+    45us) service_pool.cc:224] Handling call
0812 06:37:18.933812 (+2067505us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260812 06:37:18.934015  9799 pending_rounds.cc:107] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae: Aborting uncommitted WRITE_OP operation due to leader change: 23.3625
I20260812 06:37:18.934160  9799 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:41528 (ReqId={client: 3067f3aef2a44cc4bb7f2ccd6acf83db, seq_no=3606, attempt_no=1}) took 2068 ms. Trace:
I20260812 06:37:18.934192  9799 rpcz_store.cc:276] 0812 06:37:16.866106 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 06:37:16.866144 (+    38us) service_pool.cc:224] Handling call
0812 06:37:18.934155 (+2068011us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260812 06:37:18.934402  9799 pending_rounds.cc:107] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae: Aborting uncommitted WRITE_OP operation due to leader change: 23.3626
I20260812 06:37:18.934545  9799 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:41528 (ReqId={client: 3067f3aef2a44cc4bb7f2ccd6acf83db, seq_no=3608, attempt_no=1}) took 2068 ms. Trace:
I20260812 06:37:18.934577  9799 rpcz_store.cc:276] 0812 06:37:16.865931 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 06:37:16.866004 (+    73us) service_pool.cc:224] Handling call
0812 06:37:18.934539 (+2068535us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260812 06:37:18.934749  9799 pending_rounds.cc:107] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae: Aborting uncommitted WRITE_OP operation due to leader change: 23.3627
I20260812 06:37:18.934892  9799 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:41528 (ReqId={client: 3067f3aef2a44cc4bb7f2ccd6acf83db, seq_no=3607, attempt_no=1}) took 2068 ms. Trace:
I20260812 06:37:18.934924  9799 rpcz_store.cc:276] 0812 06:37:16.866367 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 06:37:16.866403 (+    36us) service_pool.cc:224] Handling call
0812 06:37:18.934886 (+2068483us) inbound_call.cc:177] Queueing failure response
Metrics: {}
W20260812 06:37:18.935635  9738 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:41528: Illegal state: replica 2df4308c852240859ada53d216ab69ae is not leader of this config: current role FOLLOWER
I20260812 06:37:18.935662 11131 consensus_queue.cc:1048] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2df4308c852240859ada53d216ab69ae" member_type: VOTER last_known_addr { host: "127.4.177.6" port: 34269 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3620, Last known committed idx: 3619, Time since last communication: 0.000s
I20260812 06:37:18.937847  6701 raft_consensus.cc:1275] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [term 24 FOLLOWER]: Refusing update from remote peer 222b20de5a1145c6b8a11df8374e8790: Log matching property violated. Preceding OpId in replica: term: 23 index: 3627. Preceding OpId from leader: term: 24 index: 3620. (term mismatch)
I20260812 06:37:18.937952  6701 pending_rounds.cc:85] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02: Aborting all ops after (but not including) 3619
I20260812 06:37:18.937994  6701 pending_rounds.cc:107] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02: Aborting uncommitted WRITE_OP operation due to leader change: 23.3620
I20260812 06:37:18.938189  6701 pending_rounds.cc:107] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02: Aborting uncommitted WRITE_OP operation due to leader change: 23.3621
I20260812 06:37:18.938367  6701 pending_rounds.cc:107] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02: Aborting uncommitted WRITE_OP operation due to leader change: 23.3622
I20260812 06:37:18.938464  6701 pending_rounds.cc:107] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02: Aborting uncommitted WRITE_OP operation due to leader change: 23.3623
I20260812 06:37:18.938546  6701 pending_rounds.cc:107] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02: Aborting uncommitted WRITE_OP operation due to leader change: 23.3624
I20260812 06:37:18.938625  6701 pending_rounds.cc:107] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02: Aborting uncommitted WRITE_OP operation due to leader change: 23.3625
I20260812 06:37:18.938772  6701 pending_rounds.cc:107] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02: Aborting uncommitted WRITE_OP operation due to leader change: 23.3626
I20260812 06:37:18.938952  6701 pending_rounds.cc:107] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02: Aborting uncommitted WRITE_OP operation due to leader change: 23.3627
W20260812 06:37:18.939389  9738 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:41528: Illegal state: replica 2df4308c852240859ada53d216ab69ae is not leader of this config: current role FOLLOWER
W20260812 06:37:18.939572  9744 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:41528: Illegal state: replica 2df4308c852240859ada53d216ab69ae is not leader of this config: current role FOLLOWER
W20260812 06:37:18.939841  9749 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:41528: Illegal state: replica 2df4308c852240859ada53d216ab69ae is not leader of this config: current role FOLLOWER
W20260812 06:37:18.939572  9742 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:41528: Illegal state: replica 2df4308c852240859ada53d216ab69ae is not leader of this config: current role FOLLOWER
I20260812 06:37:18.940269 11131 consensus_queue.cc:1048] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [LEADER]: Connected to new peer: Peer: permanent_uuid: "42895a9bcba64aa484f6509605035d02" member_type: VOTER last_known_addr { host: "127.4.177.4" port: 38181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3620, Last known committed idx: 3619, Time since last communication: 0.000s
W20260812 06:37:18.940891  9742 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:41528: Illegal state: replica 2df4308c852240859ada53d216ab69ae is not leader of this config: current role FOLLOWER
W20260812 06:37:18.942111  9742 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:41528: Illegal state: replica 2df4308c852240859ada53d216ab69ae is not leader of this config: current role FOLLOWER
W20260812 06:37:18.942150  9749 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:41528: Illegal state: replica 2df4308c852240859ada53d216ab69ae is not leader of this config: current role FOLLOWER
W20260812 06:37:18.943368  6623 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:42408: Illegal state: replica 42895a9bcba64aa484f6509605035d02 is not leader of this config: current role FOLLOWER
W20260812 06:37:18.944427  6623 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:42408: Illegal state: replica 42895a9bcba64aa484f6509605035d02 is not leader of this config: current role FOLLOWER
W20260812 06:37:18.945426  6623 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:42408: Illegal state: replica 42895a9bcba64aa484f6509605035d02 is not leader of this config: current role FOLLOWER
W20260812 06:37:18.946451  6623 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:42408: Illegal state: replica 42895a9bcba64aa484f6509605035d02 is not leader of this config: current role FOLLOWER
W20260812 06:37:18.947391  6623 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:42408: Illegal state: replica 42895a9bcba64aa484f6509605035d02 is not leader of this config: current role FOLLOWER
W20260812 06:37:18.948338  6623 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:42408: Illegal state: replica 42895a9bcba64aa484f6509605035d02 is not leader of this config: current role FOLLOWER
I20260812 06:37:18.949249  6238 raft_consensus.cc:1275] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b [term 24 FOLLOWER]: Refusing update from remote peer 222b20de5a1145c6b8a11df8374e8790: Log matching property violated. Preceding OpId in replica: term: 23 index: 3576. Preceding OpId from leader: term: 24 index: 3620. (index mismatch)
I20260812 06:37:18.950255  6048 raft_consensus.cc:1275] T 7065bb403e3047dd9a0853c49318ef1f P ec6c8d818e744cdbb6b0d15b321010e0 [term 24 FOLLOWER]: Refusing update from remote peer 222b20de5a1145c6b8a11df8374e8790: Log matching property violated. Preceding OpId in replica: term: 23 index: 3514. Preceding OpId from leader: term: 24 index: 3620. (index mismatch)
W20260812 06:37:18.951264  6631 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:42408: Illegal state: replica 42895a9bcba64aa484f6509605035d02 is not leader of this config: current role FOLLOWER
W20260812 06:37:18.952243  6623 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:42408: Illegal state: replica 42895a9bcba64aa484f6509605035d02 is not leader of this config: current role FOLLOWER
I20260812 06:37:18.952466 11423 consensus_queue.cc:1048] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 41057 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3620, Last known committed idx: 3502, Time since last communication: 0.000s
I20260812 06:37:18.952625 11427 consensus_queue.cc:1048] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8d90d61da62549208fe117b4384a469b" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 41539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3620, Last known committed idx: 3572, Time since last communication: 0.000s
I20260812 06:37:19.088390 11446 mvcc.cc:204] Tried to move back new op lower bound from 7317572153145311232 to 7317572150747422720. Current Snapshot: MvccSnapshot[applied={T|T < 7317572153145311232 or (T in {7317572153145311232})}]
I20260812 06:37:19.878489 11392 heartbeater.cc:499] Master 127.4.177.62:42183 was elected leader, sending a full tablet report...
I20260812 06:37:20.803092 11506 raft_consensus.cc:493] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 222b20de5a1145c6b8a11df8374e8790)
I20260812 06:37:20.803274 11506 raft_consensus.cc:515] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df4308c852240859ada53d216ab69ae" member_type: VOTER last_known_addr { host: "127.4.177.6" port: 34269 } } peers { permanent_uuid: "42895a9bcba64aa484f6509605035d02" member_type: VOTER last_known_addr { host: "127.4.177.4" port: 38181 } } peers { permanent_uuid: "222b20de5a1145c6b8a11df8374e8790" member_type: VOTER last_known_addr { host: "127.4.177.5" port: 43521 } } peers { permanent_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 40897 } } peers { permanent_uuid: "8d90d61da62549208fe117b4384a469b" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 41539 } } peers { permanent_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" member_type: VOTER last_known_addr { host: "127.4.177.7" port: 46299 } } peers { permanent_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 41057 } }
I20260812 06:37:20.804433  9799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" candidate_term: 25 candidate_status { last_received { term: 24 index: 3687 } } ignore_live_leader: false dest_uuid: "2df4308c852240859ada53d216ab69ae" is_pre_election: true
I20260812 06:37:20.804761  6700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" candidate_term: 25 candidate_status { last_received { term: 24 index: 3687 } } ignore_live_leader: false dest_uuid: "42895a9bcba64aa484f6509605035d02" is_pre_election: true
I20260812 06:37:20.804921 11506 leader_election.cc:290] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 2df4308c852240859ada53d216ab69ae (127.4.177.6:34269), 42895a9bcba64aa484f6509605035d02 (127.4.177.4:38181), 222b20de5a1145c6b8a11df8374e8790 (127.4.177.5:43521), 7962f587ccd842ad8a3f6ed9a5d7707e (127.4.177.3:40897), 8d90d61da62549208fe117b4384a469b (127.4.177.2:41539), ec6c8d818e744cdbb6b0d15b321010e0 (127.4.177.1:41057)
I20260812 06:37:20.806277  6046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" candidate_term: 25 candidate_status { last_received { term: 24 index: 3687 } } ignore_live_leader: false dest_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" is_pre_election: true
I20260812 06:37:20.806399  6236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" candidate_term: 25 candidate_status { last_received { term: 24 index: 3687 } } ignore_live_leader: false dest_uuid: "8d90d61da62549208fe117b4384a469b" is_pre_election: true
I20260812 06:37:20.806886  7211 leader_election.cc:304] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 5 responses out of 7 voters: 1 yes votes; 4 no votes. yes voters: b037d17b21f8422fa5e41b70eb4dcd08; no voters: 2df4308c852240859ada53d216ab69ae, 42895a9bcba64aa484f6509605035d02, 8d90d61da62549208fe117b4384a469b, ec6c8d818e744cdbb6b0d15b321010e0
I20260812 06:37:20.807228 11506 raft_consensus.cc:2749] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260812 06:37:20.810286 11293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" candidate_term: 25 candidate_status { last_received { term: 24 index: 3687 } } ignore_live_leader: false dest_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" is_pre_election: true
W20260812 06:37:20.811522  7211 leader_election.cc:343] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08 [CANDIDATE]: Term 25 pre-election: Tablet error from VoteRequest() call to peer 7962f587ccd842ad8a3f6ed9a5d7707e (127.4.177.3:40897): Illegal state: must be running to vote when last-logged opid is not known
I20260812 06:37:20.915028 11509 raft_consensus.cc:493] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 222b20de5a1145c6b8a11df8374e8790)
I20260812 06:37:20.915164 11509 raft_consensus.cc:515] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df4308c852240859ada53d216ab69ae" member_type: VOTER last_known_addr { host: "127.4.177.6" port: 34269 } } peers { permanent_uuid: "42895a9bcba64aa484f6509605035d02" member_type: VOTER last_known_addr { host: "127.4.177.4" port: 38181 } } peers { permanent_uuid: "222b20de5a1145c6b8a11df8374e8790" member_type: VOTER last_known_addr { host: "127.4.177.5" port: 43521 } } peers { permanent_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 40897 } } peers { permanent_uuid: "8d90d61da62549208fe117b4384a469b" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 41539 } } peers { permanent_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" member_type: VOTER last_known_addr { host: "127.4.177.7" port: 46299 } } peers { permanent_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 41057 } }
I20260812 06:37:20.915676 11512 raft_consensus.cc:493] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 222b20de5a1145c6b8a11df8374e8790)
I20260812 06:37:20.915805 11512 raft_consensus.cc:515] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df4308c852240859ada53d216ab69ae" member_type: VOTER last_known_addr { host: "127.4.177.6" port: 34269 } } peers { permanent_uuid: "42895a9bcba64aa484f6509605035d02" member_type: VOTER last_known_addr { host: "127.4.177.4" port: 38181 } } peers { permanent_uuid: "222b20de5a1145c6b8a11df8374e8790" member_type: VOTER last_known_addr { host: "127.4.177.5" port: 43521 } } peers { permanent_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 40897 } } peers { permanent_uuid: "8d90d61da62549208fe117b4384a469b" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 41539 } } peers { permanent_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" member_type: VOTER last_known_addr { host: "127.4.177.7" port: 46299 } } peers { permanent_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 41057 } }
I20260812 06:37:20.916504 11512 leader_election.cc:290] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 2df4308c852240859ada53d216ab69ae (127.4.177.6:34269), 222b20de5a1145c6b8a11df8374e8790 (127.4.177.5:43521), 7962f587ccd842ad8a3f6ed9a5d7707e (127.4.177.3:40897), 8d90d61da62549208fe117b4384a469b (127.4.177.2:41539), b037d17b21f8422fa5e41b70eb4dcd08 (127.4.177.7:46299), ec6c8d818e744cdbb6b0d15b321010e0 (127.4.177.1:41057)
I20260812 06:37:20.916690  6700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "8d90d61da62549208fe117b4384a469b" candidate_term: 25 candidate_status { last_received { term: 24 index: 3725 } } ignore_live_leader: false dest_uuid: "42895a9bcba64aa484f6509605035d02" is_pre_election: true
I20260812 06:37:20.916719 11509 leader_election.cc:290] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 2df4308c852240859ada53d216ab69ae (127.4.177.6:34269), 42895a9bcba64aa484f6509605035d02 (127.4.177.4:38181), 222b20de5a1145c6b8a11df8374e8790 (127.4.177.5:43521), 7962f587ccd842ad8a3f6ed9a5d7707e (127.4.177.3:40897), b037d17b21f8422fa5e41b70eb4dcd08 (127.4.177.7:46299), ec6c8d818e744cdbb6b0d15b321010e0 (127.4.177.1:41057)
I20260812 06:37:20.916843  6700 raft_consensus.cc:2410] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8d90d61da62549208fe117b4384a469b for term 25 because replica has last-logged OpId of term: 24 index: 3726, which is greater than that of the candidate, which has last-logged OpId of term: 24 index: 3725.
I20260812 06:37:20.917383  7321 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "42895a9bcba64aa484f6509605035d02" candidate_term: 25 candidate_status { last_received { term: 24 index: 3726 } } ignore_live_leader: false dest_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" is_pre_election: true
I20260812 06:37:20.917531  7321 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 42895a9bcba64aa484f6509605035d02 in term 24.
I20260812 06:37:20.917557  6046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "8d90d61da62549208fe117b4384a469b" candidate_term: 25 candidate_status { last_received { term: 24 index: 3725 } } ignore_live_leader: false dest_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" is_pre_election: true
I20260812 06:37:20.918088  7322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "8d90d61da62549208fe117b4384a469b" candidate_term: 25 candidate_status { last_received { term: 24 index: 3725 } } ignore_live_leader: false dest_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" is_pre_election: true
I20260812 06:37:20.918238  7322 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8d90d61da62549208fe117b4384a469b in term 24.
I20260812 06:37:20.918494  9794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "8d90d61da62549208fe117b4384a469b" candidate_term: 25 candidate_status { last_received { term: 24 index: 3725 } } ignore_live_leader: false dest_uuid: "2df4308c852240859ada53d216ab69ae" is_pre_election: true
I20260812 06:37:20.918555  9799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "42895a9bcba64aa484f6509605035d02" candidate_term: 25 candidate_status { last_received { term: 24 index: 3726 } } ignore_live_leader: false dest_uuid: "2df4308c852240859ada53d216ab69ae" is_pre_election: true
I20260812 06:37:20.918680  9794 raft_consensus.cc:2410] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8d90d61da62549208fe117b4384a469b for term 25 because replica has last-logged OpId of term: 24 index: 3726, which is greater than that of the candidate, which has last-logged OpId of term: 24 index: 3725.
I20260812 06:37:20.919126 11519 raft_consensus.cc:493] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 222b20de5a1145c6b8a11df8374e8790)
I20260812 06:37:20.919220  6046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "42895a9bcba64aa484f6509605035d02" candidate_term: 25 candidate_status { last_received { term: 24 index: 3726 } } ignore_live_leader: false dest_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" is_pre_election: true
I20260812 06:37:20.919291 11519 raft_consensus.cc:515] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df4308c852240859ada53d216ab69ae" member_type: VOTER last_known_addr { host: "127.4.177.6" port: 34269 } } peers { permanent_uuid: "42895a9bcba64aa484f6509605035d02" member_type: VOTER last_known_addr { host: "127.4.177.4" port: 38181 } } peers { permanent_uuid: "222b20de5a1145c6b8a11df8374e8790" member_type: VOTER last_known_addr { host: "127.4.177.5" port: 43521 } } peers { permanent_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 40897 } } peers { permanent_uuid: "8d90d61da62549208fe117b4384a469b" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 41539 } } peers { permanent_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" member_type: VOTER last_known_addr { host: "127.4.177.7" port: 46299 } } peers { permanent_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 41057 } }
I20260812 06:37:20.920023  6700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "2df4308c852240859ada53d216ab69ae" candidate_term: 25 candidate_status { last_received { term: 24 index: 3726 } } ignore_live_leader: false dest_uuid: "42895a9bcba64aa484f6509605035d02" is_pre_election: true
I20260812 06:37:20.920143  6700 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2df4308c852240859ada53d216ab69ae in term 24.
I20260812 06:37:20.920125  6236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "42895a9bcba64aa484f6509605035d02" candidate_term: 25 candidate_status { last_received { term: 24 index: 3726 } } ignore_live_leader: false dest_uuid: "8d90d61da62549208fe117b4384a469b" is_pre_election: true
I20260812 06:37:20.920243  6236 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 42895a9bcba64aa484f6509605035d02 in term 24.
I20260812 06:37:20.921269 11519 leader_election.cc:290] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 42895a9bcba64aa484f6509605035d02 (127.4.177.4:38181), 222b20de5a1145c6b8a11df8374e8790 (127.4.177.5:43521), 7962f587ccd842ad8a3f6ed9a5d7707e (127.4.177.3:40897), 8d90d61da62549208fe117b4384a469b (127.4.177.2:41539), b037d17b21f8422fa5e41b70eb4dcd08 (127.4.177.7:46299), ec6c8d818e744cdbb6b0d15b321010e0 (127.4.177.1:41057)
I20260812 06:37:20.921875  6046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "2df4308c852240859ada53d216ab69ae" candidate_term: 25 candidate_status { last_received { term: 24 index: 3726 } } ignore_live_leader: false dest_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" is_pre_election: true
I20260812 06:37:20.922084  7322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "2df4308c852240859ada53d216ab69ae" candidate_term: 25 candidate_status { last_received { term: 24 index: 3726 } } ignore_live_leader: false dest_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" is_pre_election: true
I20260812 06:37:20.922169  6236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "2df4308c852240859ada53d216ab69ae" candidate_term: 25 candidate_status { last_received { term: 24 index: 3726 } } ignore_live_leader: false dest_uuid: "8d90d61da62549208fe117b4384a469b" is_pre_election: true
I20260812 06:37:20.922202  7322 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2df4308c852240859ada53d216ab69ae in term 24.
I20260812 06:37:20.922297  6236 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2df4308c852240859ada53d216ab69ae in term 24.
I20260812 06:37:20.922559  9714 leader_election.cc:304] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 7 voters: 4 yes votes; 0 no votes. yes voters: 2df4308c852240859ada53d216ab69ae, 42895a9bcba64aa484f6509605035d02, 8d90d61da62549208fe117b4384a469b, b037d17b21f8422fa5e41b70eb4dcd08; no voters: 
I20260812 06:37:20.922794 11519 raft_consensus.cc:2804] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260812 06:37:20.922868 11519 raft_consensus.cc:493] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [term 24 FOLLOWER]: Starting leader election (detected failure of leader 222b20de5a1145c6b8a11df8374e8790)
I20260812 06:37:20.922915 11519 raft_consensus.cc:3060] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [term 24 FOLLOWER]: Advancing to term 25
I20260812 06:37:20.924147 11519 raft_consensus.cc:515] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df4308c852240859ada53d216ab69ae" member_type: VOTER last_known_addr { host: "127.4.177.6" port: 34269 } } peers { permanent_uuid: "42895a9bcba64aa484f6509605035d02" member_type: VOTER last_known_addr { host: "127.4.177.4" port: 38181 } } peers { permanent_uuid: "222b20de5a1145c6b8a11df8374e8790" member_type: VOTER last_known_addr { host: "127.4.177.5" port: 43521 } } peers { permanent_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 40897 } } peers { permanent_uuid: "8d90d61da62549208fe117b4384a469b" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 41539 } } peers { permanent_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" member_type: VOTER last_known_addr { host: "127.4.177.7" port: 46299 } } peers { permanent_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 41057 } }
I20260812 06:37:20.924410 11293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "42895a9bcba64aa484f6509605035d02" candidate_term: 25 candidate_status { last_received { term: 24 index: 3726 } } ignore_live_leader: false dest_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" is_pre_election: true
W20260812 06:37:20.924760  6597 leader_election.cc:343] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [CANDIDATE]: Term 25 pre-election: Tablet error from VoteRequest() call to peer 7962f587ccd842ad8a3f6ed9a5d7707e (127.4.177.3:40897): Illegal state: must be running to vote when last-logged opid is not known
I20260812 06:37:20.924924  6700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "2df4308c852240859ada53d216ab69ae" candidate_term: 25 candidate_status { last_received { term: 24 index: 3726 } } ignore_live_leader: false dest_uuid: "42895a9bcba64aa484f6509605035d02"
I20260812 06:37:20.925071  6700 raft_consensus.cc:3060] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [term 24 FOLLOWER]: Advancing to term 25
I20260812 06:37:20.925339  6236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "2df4308c852240859ada53d216ab69ae" candidate_term: 25 candidate_status { last_received { term: 24 index: 3726 } } ignore_live_leader: false dest_uuid: "8d90d61da62549208fe117b4384a469b"
I20260812 06:37:20.925416 11519 leader_election.cc:290] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [CANDIDATE]: Term 25 election: Requested vote from peers 42895a9bcba64aa484f6509605035d02 (127.4.177.4:38181), 222b20de5a1145c6b8a11df8374e8790 (127.4.177.5:43521), 7962f587ccd842ad8a3f6ed9a5d7707e (127.4.177.3:40897), 8d90d61da62549208fe117b4384a469b (127.4.177.2:41539), b037d17b21f8422fa5e41b70eb4dcd08 (127.4.177.7:46299), ec6c8d818e744cdbb6b0d15b321010e0 (127.4.177.1:41057)
I20260812 06:37:20.925441  6236 raft_consensus.cc:3060] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b [term 24 FOLLOWER]: Advancing to term 25
I20260812 06:37:20.926146  6700 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2df4308c852240859ada53d216ab69ae in term 25.
I20260812 06:37:20.926561  6236 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2df4308c852240859ada53d216ab69ae in term 25.
I20260812 06:37:20.926641  7322 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "2df4308c852240859ada53d216ab69ae" candidate_term: 25 candidate_status { last_received { term: 24 index: 3726 } } ignore_live_leader: false dest_uuid: "b037d17b21f8422fa5e41b70eb4dcd08"
I20260812 06:37:20.926743  7322 raft_consensus.cc:3060] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08 [term 24 FOLLOWER]: Advancing to term 25
I20260812 06:37:20.926889  6046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "2df4308c852240859ada53d216ab69ae" candidate_term: 25 candidate_status { last_received { term: 24 index: 3726 } } ignore_live_leader: false dest_uuid: "ec6c8d818e744cdbb6b0d15b321010e0"
I20260812 06:37:20.927758 11293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "8d90d61da62549208fe117b4384a469b" candidate_term: 25 candidate_status { last_received { term: 24 index: 3725 } } ignore_live_leader: false dest_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" is_pre_election: true
I20260812 06:37:20.927942  7322 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2df4308c852240859ada53d216ab69ae in term 25.
W20260812 06:37:20.928139  6150 leader_election.cc:343] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b [CANDIDATE]: Term 25 pre-election: Tablet error from VoteRequest() call to peer 7962f587ccd842ad8a3f6ed9a5d7707e (127.4.177.3:40897): Illegal state: must be running to vote when last-logged opid is not known
I20260812 06:37:20.928207  6150 leader_election.cc:304] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 6 responses out of 7 voters: 2 yes votes; 4 no votes. yes voters: 8d90d61da62549208fe117b4384a469b, b037d17b21f8422fa5e41b70eb4dcd08; no voters: 2df4308c852240859ada53d216ab69ae, 42895a9bcba64aa484f6509605035d02, 7962f587ccd842ad8a3f6ed9a5d7707e, ec6c8d818e744cdbb6b0d15b321010e0
I20260812 06:37:20.928253  9710 leader_election.cc:304] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 5 responses out of 7 voters: 4 yes votes; 1 no votes. yes voters: 2df4308c852240859ada53d216ab69ae, 42895a9bcba64aa484f6509605035d02, 8d90d61da62549208fe117b4384a469b, b037d17b21f8422fa5e41b70eb4dcd08; no voters: ec6c8d818e744cdbb6b0d15b321010e0
I20260812 06:37:20.928360 11509 raft_consensus.cc:2749] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b [term 25 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260812 06:37:20.928517 11519 raft_consensus.cc:2804] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [term 25 FOLLOWER]: Leader election won for term 25
I20260812 06:37:20.928711 11519 raft_consensus.cc:697] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [term 25 LEADER]: Becoming Leader. State: Replica: 2df4308c852240859ada53d216ab69ae, State: Running, Role: LEADER
I20260812 06:37:20.928929 11519 consensus_queue.cc:237] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3718, Committed index: 3718, Last appended: 24.3726, Last appended by leader: 3726, Current term: 25, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df4308c852240859ada53d216ab69ae" member_type: VOTER last_known_addr { host: "127.4.177.6" port: 34269 } } peers { permanent_uuid: "42895a9bcba64aa484f6509605035d02" member_type: VOTER last_known_addr { host: "127.4.177.4" port: 38181 } } peers { permanent_uuid: "222b20de5a1145c6b8a11df8374e8790" member_type: VOTER last_known_addr { host: "127.4.177.5" port: 43521 } } peers { permanent_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 40897 } } peers { permanent_uuid: "8d90d61da62549208fe117b4384a469b" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 41539 } } peers { permanent_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" member_type: VOTER last_known_addr { host: "127.4.177.7" port: 46299 } } peers { permanent_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 41057 } }
I20260812 06:37:20.930015 11293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "2df4308c852240859ada53d216ab69ae" candidate_term: 25 candidate_status { last_received { term: 24 index: 3726 } } ignore_live_leader: false dest_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" is_pre_election: true
I20260812 06:37:20.930104 11291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "2df4308c852240859ada53d216ab69ae" candidate_term: 25 candidate_status { last_received { term: 24 index: 3726 } } ignore_live_leader: false dest_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e"
W20260812 06:37:20.930446  9714 leader_election.cc:343] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [CANDIDATE]: Term 25 pre-election: Tablet error from VoteRequest() call to peer 7962f587ccd842ad8a3f6ed9a5d7707e (127.4.177.3:40897): Illegal state: must be running to vote when last-logged opid is not known
W20260812 06:37:20.930583  9714 leader_election.cc:343] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [CANDIDATE]: Term 25 election: Tablet error from VoteRequest() call to peer 7962f587ccd842ad8a3f6ed9a5d7707e (127.4.177.3:40897): Illegal state: must be running to vote when last-logged opid is not known
I20260812 06:37:20.931821  5808 catalog_manager.cc:5719] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae reported cstate change: term changed from 24 to 25, leader changed from 222b20de5a1145c6b8a11df8374e8790 (127.4.177.5) to 2df4308c852240859ada53d216ab69ae (127.4.177.6). New cstate: current_term: 25 leader_uuid: "2df4308c852240859ada53d216ab69ae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df4308c852240859ada53d216ab69ae" member_type: VOTER last_known_addr { host: "127.4.177.6" port: 34269 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "42895a9bcba64aa484f6509605035d02" member_type: VOTER last_known_addr { host: "127.4.177.4" port: 38181 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "222b20de5a1145c6b8a11df8374e8790" member_type: VOTER last_known_addr { host: "127.4.177.5" port: 43521 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 40897 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8d90d61da62549208fe117b4384a469b" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 41539 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" member_type: VOTER last_known_addr { host: "127.4.177.7" port: 46299 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 41057 } health_report { overall_health: UNKNOWN } } }
W20260812 06:37:21.113401  9315 connection.cc:256] RPC call timeout handler was delayed by -1.0103772222064435s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1s for a response
I20260812 06:37:21.114742  9414 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "42895a9bcba64aa484f6509605035d02" candidate_term: 25 candidate_status { last_received { term: 24 index: 3726 } } ignore_live_leader: false dest_uuid: "222b20de5a1145c6b8a11df8374e8790" is_pre_election: true
I20260812 06:37:21.115413  6597 leader_election.cc:304] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 7 responses out of 7 voters: 3 yes votes; 4 no votes. yes voters: 42895a9bcba64aa484f6509605035d02, 8d90d61da62549208fe117b4384a469b, b037d17b21f8422fa5e41b70eb4dcd08; no voters: 222b20de5a1145c6b8a11df8374e8790, 2df4308c852240859ada53d216ab69ae, 7962f587ccd842ad8a3f6ed9a5d7707e, ec6c8d818e744cdbb6b0d15b321010e0
I20260812 06:37:21.115794  9420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "8d90d61da62549208fe117b4384a469b" candidate_term: 25 candidate_status { last_received { term: 24 index: 3725 } } ignore_live_leader: false dest_uuid: "222b20de5a1145c6b8a11df8374e8790" is_pre_election: true
I20260812 06:37:21.115953  9794 raft_consensus.cc:1240] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [term 25 LEADER]: Rejecting Update request from peer 222b20de5a1145c6b8a11df8374e8790 for earlier term 24. Current term is 25. Ops: []
I20260812 06:37:21.116068  9421 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" candidate_term: 25 candidate_status { last_received { term: 24 index: 3687 } } ignore_live_leader: false dest_uuid: "222b20de5a1145c6b8a11df8374e8790" is_pre_election: true
I20260812 06:37:21.117126 11470 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:46864 (ReqId={client: 3067f3aef2a44cc4bb7f2ccd6acf83db, seq_no=3699, attempt_no=0}) took 1718 ms. Trace:
I20260812 06:37:21.117882 11539 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:46864 (ReqId={client: 3067f3aef2a44cc4bb7f2ccd6acf83db, seq_no=3700, attempt_no=0}) took 1718 ms. Trace:
I20260812 06:37:21.117208 11470 rpcz_store.cc:276] 0812 06:37:19.398999 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 06:37:19.399257 (+   258us) service_pool.cc:224] Handling call
0812 06:37:21.117101 (+1717844us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0812 06:37:19.408370 (+     0us) write_op.cc:183] PREPARE: starting on tablet 7065bb403e3047dd9a0853c49318ef1f
0812 06:37:19.408462 (+    92us) write_op.cc:432] Acquiring schema lock in shared mode
0812 06:37:19.408466 (+     4us) write_op.cc:435] Acquired schema lock
0812 06:37:19.408467 (+     1us) tablet.cc:662] Decoding operations
0812 06:37:19.408570 (+   103us) write_op.cc:620] Acquiring the partition lock for write op
0812 06:37:19.408577 (+     7us) write_op.cc:641] Partition lock acquired for write op
0812 06:37:19.408582 (+     5us) tablet.cc:685] Acquiring locks for 10 operations
0812 06:37:19.408636 (+    54us) tablet.cc:701] Row locks acquired
0812 06:37:19.408637 (+     1us) write_op.cc:260] PREPARE: finished
0812 06:37:19.408680 (+    43us) write_op.cc:270] Start()
0812 06:37:19.408703 (+    23us) write_op.cc:276] Timestamp: P: 1786516639408587 usec, L: 0
0812 06:37:19.408705 (+     2us) op_driver.cc:348] REPLICATION: starting
0812 06:37:19.408824 (+   119us) log.cc:844] Serialized 686 byte log entry
0812 06:37:21.115752 (+1706928us) op_driver.cc:464] REPLICATION: finished
0812 06:37:21.116405 (+   653us) write_op.cc:301] APPLY: starting
0812 06:37:21.116432 (+    27us) tablet.cc:1367] starting BulkCheckPresence
0812 06:37:21.116473 (+    41us) tablet.cc:1370] finished BulkCheckPresence
0812 06:37:21.116485 (+    12us) tablet.cc:1372] starting ApplyRowOperation cycle
0812 06:37:21.116704 (+   219us) tablet.cc:1383] finished ApplyRowOperation cycle
0812 06:37:21.116749 (+    45us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0812 06:37:21.116755 (+     6us) write_op.cc:312] APPLY: finished
0812 06:37:21.116831 (+    76us) log.cc:844] Serialized 77 byte log entry
0812 06:37:21.117000 (+   169us) write_op.cc:489] Releasing partition, row and schema locks
0812 06:37:21.117019 (+    19us) write_op.cc:454] Released schema lock
0812 06:37:21.117026 (+     7us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":608,"num_ops":10,"prepare.queue_time_us":8781,"prepare.run_cpu_time_us":568,"prepare.run_wall_time_us":1126,"raft.queue_time_us":264,"raft.run_cpu_time_us":258,"raft.run_wall_time_us":259,"replication_time_us":1707016,"thread_start_us":85,"threads_started":1}]]}
I20260812 06:37:21.117951 11539 rpcz_store.cc:276] 0812 06:37:19.399083 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 06:37:19.399130 (+    47us) service_pool.cc:224] Handling call
0812 06:37:21.117854 (+1718724us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0812 06:37:19.409516 (+     0us) write_op.cc:183] PREPARE: starting on tablet 7065bb403e3047dd9a0853c49318ef1f
0812 06:37:19.409592 (+    76us) write_op.cc:432] Acquiring schema lock in shared mode
0812 06:37:19.409595 (+     3us) write_op.cc:435] Acquired schema lock
0812 06:37:19.409597 (+     2us) tablet.cc:662] Decoding operations
0812 06:37:19.409689 (+    92us) write_op.cc:620] Acquiring the partition lock for write op
0812 06:37:19.409697 (+     8us) write_op.cc:641] Partition lock acquired for write op
0812 06:37:19.409701 (+     4us) tablet.cc:685] Acquiring locks for 10 operations
0812 06:37:19.409771 (+    70us) tablet.cc:701] Row locks acquired
0812 06:37:19.409774 (+     3us) write_op.cc:260] PREPARE: finished
0812 06:37:19.409804 (+    30us) write_op.cc:270] Start()
0812 06:37:19.409825 (+    21us) write_op.cc:276] Timestamp: P: 1786516639409729 usec, L: 0
0812 06:37:19.409827 (+     2us) op_driver.cc:348] REPLICATION: starting
0812 06:37:19.409955 (+   128us) log.cc:844] Serialized 696 byte log entry
0812 06:37:21.115804 (+1705849us) op_driver.cc:464] REPLICATION: finished
0812 06:37:21.117421 (+  1617us) write_op.cc:301] APPLY: starting
0812 06:37:21.117433 (+    12us) tablet.cc:1367] starting BulkCheckPresence
0812 06:37:21.117465 (+    32us) tablet.cc:1370] finished BulkCheckPresence
0812 06:37:21.117477 (+    12us) tablet.cc:1372] starting ApplyRowOperation cycle
0812 06:37:21.117656 (+   179us) tablet.cc:1383] finished ApplyRowOperation cycle
0812 06:37:21.117684 (+    28us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0812 06:37:21.117688 (+     4us) write_op.cc:312] APPLY: finished
0812 06:37:21.117743 (+    55us) log.cc:844] Serialized 77 byte log entry
0812 06:37:21.117779 (+    36us) write_op.cc:489] Releasing partition, row and schema locks
0812 06:37:21.117792 (+    13us) write_op.cc:454] Released schema lock
0812 06:37:21.117796 (+     4us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"spinlock_wait_cycles":8960,"child_traces":[["op",{"apply.queue_time_us":1575,"mutex_wait_us":29,"num_ops":10,"prepare.queue_time_us":9934,"prepare.run_cpu_time_us":517,"prepare.run_wall_time_us":710,"raft.queue_time_us":59,"raft.run_cpu_time_us":4,"raft.run_wall_time_us":5,"replication_time_us":1705963,"thread_start_us":107,"threads_started":1}]]}
I20260812 06:37:21.118822 11470 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:46864 (ReqId={client: 3067f3aef2a44cc4bb7f2ccd6acf83db, seq_no=3701, attempt_no=0}) took 1719 ms. Trace:
I20260812 06:37:21.118927 11512 raft_consensus.cc:2749] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [term 25 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260812 06:37:21.118888 11470 rpcz_store.cc:276] 0812 06:37:19.399154 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 06:37:19.399619 (+   465us) service_pool.cc:224] Handling call
0812 06:37:21.118812 (+1719193us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0812 06:37:19.410268 (+     0us) write_op.cc:183] PREPARE: starting on tablet 7065bb403e3047dd9a0853c49318ef1f
0812 06:37:19.410344 (+    76us) write_op.cc:432] Acquiring schema lock in shared mode
0812 06:37:19.410348 (+     4us) write_op.cc:435] Acquired schema lock
0812 06:37:19.410349 (+     1us) tablet.cc:662] Decoding operations
0812 06:37:19.410432 (+    83us) write_op.cc:620] Acquiring the partition lock for write op
0812 06:37:19.410438 (+     6us) write_op.cc:641] Partition lock acquired for write op
0812 06:37:19.410441 (+     3us) tablet.cc:685] Acquiring locks for 10 operations
0812 06:37:19.410497 (+    56us) tablet.cc:701] Row locks acquired
0812 06:37:19.410499 (+     2us) write_op.cc:260] PREPARE: finished
0812 06:37:19.410525 (+    26us) write_op.cc:270] Start()
0812 06:37:19.410540 (+    15us) write_op.cc:276] Timestamp: P: 1786516639410450 usec, L: 0
0812 06:37:19.410541 (+     1us) op_driver.cc:348] REPLICATION: starting
0812 06:37:19.410641 (+   100us) log.cc:844] Serialized 696 byte log entry
0812 06:37:21.115972 (+1705331us) op_driver.cc:464] REPLICATION: finished
0812 06:37:21.118252 (+  2280us) write_op.cc:301] APPLY: starting
0812 06:37:21.118263 (+    11us) tablet.cc:1367] starting BulkCheckPresence
0812 06:37:21.118293 (+    30us) tablet.cc:1370] finished BulkCheckPresence
0812 06:37:21.118306 (+    13us) tablet.cc:1372] starting ApplyRowOperation cycle
0812 06:37:21.118537 (+   231us) tablet.cc:1383] finished ApplyRowOperation cycle
0812 06:37:21.118558 (+    21us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0812 06:37:21.118562 (+     4us) write_op.cc:312] APPLY: finished
0812 06:37:21.118607 (+    45us) log.cc:844] Serialized 77 byte log entry
0812 06:37:21.118730 (+   123us) write_op.cc:489] Releasing partition, row and schema locks
0812 06:37:21.118750 (+    20us) write_op.cc:454] Released schema lock
0812 06:37:21.118755 (+     5us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":2218,"num_ops":10,"prepare.queue_time_us":10367,"prepare.run_cpu_time_us":445,"prepare.run_wall_time_us":446,"raft.queue_time_us":24,"raft.run_cpu_time_us":5,"raft.run_wall_time_us":6,"replication_time_us":1705414,"spinlock_wait_cycles":2304,"thread_start_us":64,"threads_started":1}]]}
I20260812 06:37:21.119227  6700 raft_consensus.cc:1240] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [term 25 FOLLOWER]: Rejecting Update request from peer 222b20de5a1145c6b8a11df8374e8790 for earlier term 24. Current term is 25. Ops: []
I20260812 06:37:21.119678  9425 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "2df4308c852240859ada53d216ab69ae" candidate_term: 25 candidate_status { last_received { term: 24 index: 3726 } } ignore_live_leader: false dest_uuid: "222b20de5a1145c6b8a11df8374e8790" is_pre_election: true
I20260812 06:37:21.119796 11470 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:46864 (ReqId={client: 3067f3aef2a44cc4bb7f2ccd6acf83db, seq_no=3703, attempt_no=0}) took 1712 ms. Trace:
I20260812 06:37:21.119850 11470 rpcz_store.cc:276] 0812 06:37:19.406967 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 06:37:19.407040 (+    73us) service_pool.cc:224] Handling call
0812 06:37:21.119787 (+1712747us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0812 06:37:19.411215 (+     0us) write_op.cc:183] PREPARE: starting on tablet 7065bb403e3047dd9a0853c49318ef1f
0812 06:37:19.411290 (+    75us) write_op.cc:432] Acquiring schema lock in shared mode
0812 06:37:19.411295 (+     5us) write_op.cc:435] Acquired schema lock
0812 06:37:19.411297 (+     2us) tablet.cc:662] Decoding operations
0812 06:37:19.411383 (+    86us) write_op.cc:620] Acquiring the partition lock for write op
0812 06:37:19.411390 (+     7us) write_op.cc:641] Partition lock acquired for write op
0812 06:37:19.411393 (+     3us) tablet.cc:685] Acquiring locks for 10 operations
0812 06:37:19.411458 (+    65us) tablet.cc:701] Row locks acquired
0812 06:37:19.411460 (+     2us) write_op.cc:260] PREPARE: finished
0812 06:37:19.411487 (+    27us) write_op.cc:270] Start()
0812 06:37:19.411504 (+    17us) write_op.cc:276] Timestamp: P: 1786516639411413 usec, L: 0
0812 06:37:19.411506 (+     2us) op_driver.cc:348] REPLICATION: starting
0812 06:37:19.411610 (+   104us) log.cc:844] Serialized 686 byte log entry
0812 06:37:21.116208 (+1704598us) op_driver.cc:464] REPLICATION: finished
0812 06:37:21.119277 (+  3069us) write_op.cc:301] APPLY: starting
0812 06:37:21.119289 (+    12us) tablet.cc:1367] starting BulkCheckPresence
0812 06:37:21.119317 (+    28us) tablet.cc:1370] finished BulkCheckPresence
0812 06:37:21.119329 (+    12us) tablet.cc:1372] starting ApplyRowOperation cycle
0812 06:37:21.119579 (+   250us) tablet.cc:1383] finished ApplyRowOperation cycle
0812 06:37:21.119618 (+    39us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0812 06:37:21.119623 (+     5us) write_op.cc:312] APPLY: finished
0812 06:37:21.119679 (+    56us) log.cc:844] Serialized 77 byte log entry
0812 06:37:21.119715 (+    36us) write_op.cc:489] Releasing partition, row and schema locks
0812 06:37:21.119728 (+    13us) write_op.cc:454] Released schema lock
0812 06:37:21.119733 (+     5us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":3039,"num_ops":10,"prepare.queue_time_us":3870,"prepare.run_cpu_time_us":469,"prepare.run_wall_time_us":470,"raft.queue_time_us":26,"raft.run_cpu_time_us":6,"raft.run_wall_time_us":6,"replication_time_us":1704688,"spinlock_wait_cycles":5888}]]}
I20260812 06:37:21.120309  9423 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "2df4308c852240859ada53d216ab69ae" candidate_term: 25 candidate_status { last_received { term: 24 index: 3726 } } ignore_live_leader: false dest_uuid: "222b20de5a1145c6b8a11df8374e8790"
I20260812 06:37:21.120770 11470 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:46864 (ReqId={client: 3067f3aef2a44cc4bb7f2ccd6acf83db, seq_no=3704, attempt_no=0}) took 1713 ms. Trace:
I20260812 06:37:21.120837 11470 rpcz_store.cc:276] 0812 06:37:19.407059 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 06:37:19.407221 (+   162us) service_pool.cc:224] Handling call
0812 06:37:21.120758 (+1713537us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0812 06:37:19.411720 (+     0us) write_op.cc:183] PREPARE: starting on tablet 7065bb403e3047dd9a0853c49318ef1f
0812 06:37:19.411793 (+    73us) write_op.cc:432] Acquiring schema lock in shared mode
0812 06:37:19.411799 (+     6us) write_op.cc:435] Acquired schema lock
0812 06:37:19.411802 (+     3us) tablet.cc:662] Decoding operations
0812 06:37:19.411884 (+    82us) write_op.cc:620] Acquiring the partition lock for write op
0812 06:37:19.411889 (+     5us) write_op.cc:641] Partition lock acquired for write op
0812 06:37:19.411892 (+     3us) tablet.cc:685] Acquiring locks for 10 operations
0812 06:37:19.411963 (+    71us) tablet.cc:701] Row locks acquired
0812 06:37:19.411965 (+     2us) write_op.cc:260] PREPARE: finished
0812 06:37:19.411994 (+    29us) write_op.cc:270] Start()
0812 06:37:19.412011 (+    17us) write_op.cc:276] Timestamp: P: 1786516639411920 usec, L: 0
0812 06:37:19.412013 (+     2us) op_driver.cc:348] REPLICATION: starting
0812 06:37:19.412107 (+    94us) log.cc:844] Serialized 696 byte log entry
0812 06:37:21.116245 (+1704138us) op_driver.cc:464] REPLICATION: finished
0812 06:37:21.120216 (+  3971us) write_op.cc:301] APPLY: starting
0812 06:37:21.120228 (+    12us) tablet.cc:1367] starting BulkCheckPresence
0812 06:37:21.120261 (+    33us) tablet.cc:1370] finished BulkCheckPresence
0812 06:37:21.120272 (+    11us) tablet.cc:1372] starting ApplyRowOperation cycle
0812 06:37:21.120489 (+   217us) tablet.cc:1383] finished ApplyRowOperation cycle
0812 06:37:21.120530 (+    41us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0812 06:37:21.120534 (+     4us) write_op.cc:312] APPLY: finished
0812 06:37:21.120606 (+    72us) log.cc:844] Serialized 77 byte log entry
0812 06:37:21.120672 (+    66us) write_op.cc:489] Releasing partition, row and schema locks
0812 06:37:21.120687 (+    15us) write_op.cc:454] Released schema lock
0812 06:37:21.120694 (+     7us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":3940,"num_ops":10,"prepare.queue_time_us":4253,"prepare.run_cpu_time_us":462,"prepare.run_wall_time_us":461,"raft.queue_time_us":29,"raft.run_cpu_time_us":5,"raft.run_wall_time_us":6,"replication_time_us":1704218}]]}
I20260812 06:37:21.124164  6236 raft_consensus.cc:1240] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b [term 25 FOLLOWER]: Rejecting Update request from peer 222b20de5a1145c6b8a11df8374e8790 for earlier term 24. Current term is 25. Ops: [24.3726-24.3726]
I20260812 06:37:21.125774  7322 raft_consensus.cc:1240] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08 [term 25 FOLLOWER]: Rejecting Update request from peer 222b20de5a1145c6b8a11df8374e8790 for earlier term 24. Current term is 25. Ops: [24.3688-24.3726]
I20260812 06:37:21.127676 11470 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:46864 (ReqId={client: 3067f3aef2a44cc4bb7f2ccd6acf83db, seq_no=3706, attempt_no=0}) took 1720 ms. Trace:
I20260812 06:37:21.127781 11470 rpcz_store.cc:276] 0812 06:37:19.407179 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 06:37:19.407497 (+   318us) service_pool.cc:224] Handling call
0812 06:37:21.127663 (+1720166us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0812 06:37:19.412210 (+     0us) write_op.cc:183] PREPARE: starting on tablet 7065bb403e3047dd9a0853c49318ef1f
0812 06:37:19.412278 (+    68us) write_op.cc:432] Acquiring schema lock in shared mode
0812 06:37:19.412281 (+     3us) write_op.cc:435] Acquired schema lock
0812 06:37:19.412282 (+     1us) tablet.cc:662] Decoding operations
0812 06:37:19.412357 (+    75us) write_op.cc:620] Acquiring the partition lock for write op
0812 06:37:19.412362 (+     5us) write_op.cc:641] Partition lock acquired for write op
0812 06:37:19.412364 (+     2us) tablet.cc:685] Acquiring locks for 10 operations
0812 06:37:19.412425 (+    61us) tablet.cc:701] Row locks acquired
0812 06:37:19.412427 (+     2us) write_op.cc:260] PREPARE: finished
0812 06:37:19.412454 (+    27us) write_op.cc:270] Start()
0812 06:37:19.412470 (+    16us) write_op.cc:276] Timestamp: P: 1786516639412380 usec, L: 0
0812 06:37:19.412472 (+     2us) op_driver.cc:348] REPLICATION: starting
0812 06:37:19.412557 (+    85us) log.cc:844] Serialized 696 byte log entry
0812 06:37:21.116291 (+1703734us) op_driver.cc:464] REPLICATION: finished
0812 06:37:21.127138 (+ 10847us) write_op.cc:301] APPLY: starting
0812 06:37:21.127150 (+    12us) tablet.cc:1367] starting BulkCheckPresence
0812 06:37:21.127189 (+    39us) tablet.cc:1370] finished BulkCheckPresence
0812 06:37:21.127201 (+    12us) tablet.cc:1372] starting ApplyRowOperation cycle
0812 06:37:21.127435 (+   234us) tablet.cc:1383] finished ApplyRowOperation cycle
0812 06:37:21.127469 (+    34us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0812 06:37:21.127474 (+     5us) write_op.cc:312] APPLY: finished
0812 06:37:21.127534 (+    60us) log.cc:844] Serialized 77 byte log entry
0812 06:37:21.127571 (+    37us) write_op.cc:489] Releasing partition, row and schema locks
0812 06:37:21.127589 (+    18us) write_op.cc:454] Released schema lock
0812 06:37:21.127599 (+    10us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":10815,"num_ops":10,"prepare.queue_time_us":4482,"prepare.run_cpu_time_us":417,"prepare.run_wall_time_us":416,"raft.queue_time_us":25,"raft.run_cpu_time_us":5,"raft.run_wall_time_us":5,"replication_time_us":1703803,"spinlock_wait_cycles":1792}]]}
I20260812 06:37:21.128827 11539 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:46864 (ReqId={client: 3067f3aef2a44cc4bb7f2ccd6acf83db, seq_no=3702, attempt_no=0}) took 1729 ms. Trace:
I20260812 06:37:21.128923 11539 rpcz_store.cc:276] 0812 06:37:19.399215 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 06:37:19.399924 (+   709us) service_pool.cc:224] Handling call
0812 06:37:21.128815 (+1728891us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0812 06:37:19.410739 (+     0us) write_op.cc:183] PREPARE: starting on tablet 7065bb403e3047dd9a0853c49318ef1f
0812 06:37:19.410813 (+    74us) write_op.cc:432] Acquiring schema lock in shared mode
0812 06:37:19.410816 (+     3us) write_op.cc:435] Acquired schema lock
0812 06:37:19.410818 (+     2us) tablet.cc:662] Decoding operations
0812 06:37:19.410896 (+    78us) write_op.cc:620] Acquiring the partition lock for write op
0812 06:37:19.410901 (+     5us) write_op.cc:641] Partition lock acquired for write op
0812 06:37:19.410904 (+     3us) tablet.cc:685] Acquiring locks for 10 operations
0812 06:37:19.410973 (+    69us) tablet.cc:701] Row locks acquired
0812 06:37:19.410975 (+     2us) write_op.cc:260] PREPARE: finished
0812 06:37:19.411001 (+    26us) write_op.cc:270] Start()
0812 06:37:19.411018 (+    17us) write_op.cc:276] Timestamp: P: 1786516639410927 usec, L: 0
0812 06:37:19.411020 (+     2us) op_driver.cc:348] REPLICATION: starting
0812 06:37:19.411122 (+   102us) log.cc:844] Serialized 696 byte log entry
0812 06:37:21.116100 (+1704978us) op_driver.cc:464] REPLICATION: finished
0812 06:37:21.118306 (+  2206us) write_op.cc:301] APPLY: starting
0812 06:37:21.118314 (+     8us) tablet.cc:1367] starting BulkCheckPresence
0812 06:37:21.118337 (+    23us) tablet.cc:1370] finished BulkCheckPresence
0812 06:37:21.118346 (+     9us) tablet.cc:1372] starting ApplyRowOperation cycle
0812 06:37:21.118521 (+   175us) tablet.cc:1383] finished ApplyRowOperation cycle
0812 06:37:21.118548 (+    27us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0812 06:37:21.118553 (+     5us) write_op.cc:312] APPLY: finished
0812 06:37:21.118607 (+    54us) log.cc:844] Serialized 77 byte log entry
0812 06:37:21.128739 (+ 10132us) write_op.cc:489] Releasing partition, row and schema locks
0812 06:37:21.128753 (+    14us) write_op.cc:454] Released schema lock
0812 06:37:21.128759 (+     6us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":2178,"mutex_wait_us":10058,"num_ops":10,"prepare.queue_time_us":10550,"prepare.run_cpu_time_us":454,"prepare.run_wall_time_us":454,"raft.queue_time_us":210,"raft.run_cpu_time_us":6,"raft.run_wall_time_us":7,"replication_time_us":1705062,"thread_start_us":56,"threads_started":1}]]}
I20260812 06:37:21.125262 11453 consensus_queue.cc:1059] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8d90d61da62549208fe117b4384a469b" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 41539 }, Status: INVALID_TERM, Last received: 24.3725, Next index: 3726, Last known committed idx: 3718, Time since last communication: 0.000s
I20260812 06:37:21.131434 11450 consensus_queue.cc:1059] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "42895a9bcba64aa484f6509605035d02" member_type: VOTER last_known_addr { host: "127.4.177.4" port: 38181 }, Status: INVALID_TERM, Last received: 24.3726, Next index: 3727, Last known committed idx: 3718, Time since last communication: 0.000s
I20260812 06:37:21.131865 11447 consensus_queue.cc:1059] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2df4308c852240859ada53d216ab69ae" member_type: VOTER last_known_addr { host: "127.4.177.6" port: 34269 }, Status: INVALID_TERM, Last received: 24.3726, Next index: 3727, Last known committed idx: 3718, Time since last communication: 0.000s
I20260812 06:37:21.132030 11447 consensus_queue.cc:1243] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [LEADER]: Peer 2df4308c852240859ada53d216ab69ae log is divergent from this leader: its last log entry 25.3727 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3718
I20260812 06:37:21.132251 11462 consensus_queue.cc:1059] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" member_type: VOTER last_known_addr { host: "127.4.177.7" port: 46299 }, Status: INVALID_TERM, Last received: 24.3687, Next index: 3688, Last known committed idx: 3684, Time since last communication: 0.000s
I20260812 06:37:21.135794 11453 raft_consensus.cc:3055] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [term 24 LEADER]: Stepping down as leader of term 24
I20260812 06:37:21.135870 11453 raft_consensus.cc:740] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 222b20de5a1145c6b8a11df8374e8790, State: Running, Role: LEADER
I20260812 06:37:21.136106 11453 consensus_queue.cc:260] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3725, Committed index: 3725, Last appended: 24.3727, Last appended by leader: 3727, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df4308c852240859ada53d216ab69ae" member_type: VOTER last_known_addr { host: "127.4.177.6" port: 34269 } } peers { permanent_uuid: "42895a9bcba64aa484f6509605035d02" member_type: VOTER last_known_addr { host: "127.4.177.4" port: 38181 } } peers { permanent_uuid: "222b20de5a1145c6b8a11df8374e8790" member_type: VOTER last_known_addr { host: "127.4.177.5" port: 43521 } } peers { permanent_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 40897 } } peers { permanent_uuid: "8d90d61da62549208fe117b4384a469b" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 41539 } } peers { permanent_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" member_type: VOTER last_known_addr { host: "127.4.177.7" port: 46299 } } peers { permanent_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 41057 } }
I20260812 06:37:21.136574 11453 raft_consensus.cc:3060] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [term 24 FOLLOWER]: Advancing to term 25
W20260812 06:37:21.138181 11543 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:46864: Illegal state: Cannot assign timestamp to op. Tablet is not in leader mode. Last heard from a leader: 0.002s ago.
W20260812 06:37:21.139242 11543 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:46864: Illegal state: Cannot assign timestamp to op. Tablet is not in leader mode. Last heard from a leader: 0.004s ago.
W20260812 06:37:21.140144 11543 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:46864: Illegal state: Cannot assign timestamp to op. Tablet is not in leader mode. Last heard from a leader: 0.004s ago.
W20260812 06:37:21.140956 11543 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:46864: Illegal state: Cannot assign timestamp to op. Tablet is not in leader mode. Last heard from a leader: 0.005s ago.
W20260812 06:37:21.141896 11543 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:46864: Illegal state: Cannot assign timestamp to op. Tablet is not in leader mode. Last heard from a leader: 0.006s ago.
W20260812 06:37:21.142881 11543 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:46864: Illegal state: Cannot assign timestamp to op. Tablet is not in leader mode. Last heard from a leader: 0.007s ago.
I20260812 06:37:21.157262  7322 raft_consensus.cc:1275] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08 [term 25 FOLLOWER]: Refusing update from remote peer 2df4308c852240859ada53d216ab69ae: Log matching property violated. Preceding OpId in replica: term: 24 index: 3687. Preceding OpId from leader: term: 25 index: 3728. (index mismatch)
I20260812 06:37:21.157933 11519 consensus_queue.cc:1048] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" member_type: VOTER last_known_addr { host: "127.4.177.7" port: 46299 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3727, Last known committed idx: 3684, Time since last communication: 0.000s
I20260812 06:37:21.159408  6046 raft_consensus.cc:3060] T 7065bb403e3047dd9a0853c49318ef1f P ec6c8d818e744cdbb6b0d15b321010e0 [term 24 FOLLOWER]: Advancing to term 25
I20260812 06:37:21.158643  6236 raft_consensus.cc:1275] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b [term 25 FOLLOWER]: Refusing update from remote peer 2df4308c852240859ada53d216ab69ae: Log matching property violated. Preceding OpId in replica: term: 24 index: 3725. Preceding OpId from leader: term: 25 index: 3728. (index mismatch)
I20260812 06:37:21.160585  9423 raft_consensus.cc:1275] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [term 25 FOLLOWER]: Refusing update from remote peer 2df4308c852240859ada53d216ab69ae: Log matching property violated. Preceding OpId in replica: term: 24 index: 3727. Preceding OpId from leader: term: 25 index: 3728. (index mismatch)
I20260812 06:37:21.160708  6700 raft_consensus.cc:1275] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [term 25 FOLLOWER]: Refusing update from remote peer 2df4308c852240859ada53d216ab69ae: Log matching property violated. Preceding OpId in replica: term: 24 index: 3726. Preceding OpId from leader: term: 25 index: 3728. (index mismatch)
I20260812 06:37:21.161427  6046 raft_consensus.cc:1275] T 7065bb403e3047dd9a0853c49318ef1f P ec6c8d818e744cdbb6b0d15b321010e0 [term 25 FOLLOWER]: Refusing update from remote peer 2df4308c852240859ada53d216ab69ae: Log matching property violated. Preceding OpId in replica: term: 24 index: 3726. Preceding OpId from leader: term: 25 index: 3728. (index mismatch)
W20260812 06:37:21.162065  9714 consensus_peers.cc:597] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae -> Peer 7962f587ccd842ad8a3f6ed9a5d7707e (127.4.177.3:40897): Couldn't send request to peer 7962f587ccd842ad8a3f6ed9a5d7707e. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260812 06:37:21.162781 11526 consensus_queue.cc:1048] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 41057 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3727, Last known committed idx: 3725, Time since last communication: 0.000s
I20260812 06:37:21.163151 11526 consensus_queue.cc:1048] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "42895a9bcba64aa484f6509605035d02" member_type: VOTER last_known_addr { host: "127.4.177.4" port: 38181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3727, Last known committed idx: 3718, Time since last communication: 0.000s
I20260812 06:37:21.163442 11526 consensus_queue.cc:1048] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "222b20de5a1145c6b8a11df8374e8790" member_type: VOTER last_known_addr { host: "127.4.177.5" port: 43521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3727, Last known committed idx: 3725, Time since last communication: 0.000s
I20260812 06:37:21.163537 11526 consensus_queue.cc:1243] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [LEADER]: Peer 222b20de5a1145c6b8a11df8374e8790 log is divergent from this leader: its last log entry 24.3727 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3725
I20260812 06:37:21.163771 11526 consensus_queue.cc:1048] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "8d90d61da62549208fe117b4384a469b" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 41539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3727, Last known committed idx: 3718, Time since last communication: 0.000s
I20260812 06:37:21.169235  9423 raft_consensus.cc:1217] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [term 25 FOLLOWER]: Deduplicated request from leader. Original: 24.3725->[24.3726-25.3733]   Dedup: 24.3726->[25.3727-25.3733]
I20260812 06:37:21.169363  9423 pending_rounds.cc:85] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790: Aborting all ops after (but not including) 3726
I20260812 06:37:21.169392  9423 pending_rounds.cc:107] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790: Aborting uncommitted WRITE_OP operation due to leader change: 24.3727
W20260812 06:37:21.169519  9423 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:46864: Aborted: Op aborted by new leader
I20260812 06:37:21.184216 11538 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:46864 (ReqId={client: 3067f3aef2a44cc4bb7f2ccd6acf83db, seq_no=3705, attempt_no=0}) took 1777 ms. Trace:
I20260812 06:37:21.184320 11538 rpcz_store.cc:276] 0812 06:37:19.407124 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 06:37:19.407750 (+   626us) service_pool.cc:224] Handling call
0812 06:37:21.184178 (+1776428us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0812 06:37:19.412650 (+     0us) write_op.cc:183] PREPARE: starting on tablet 7065bb403e3047dd9a0853c49318ef1f
0812 06:37:19.412723 (+    73us) write_op.cc:432] Acquiring schema lock in shared mode
0812 06:37:19.412726 (+     3us) write_op.cc:435] Acquired schema lock
0812 06:37:19.412727 (+     1us) tablet.cc:662] Decoding operations
0812 06:37:19.412819 (+    92us) write_op.cc:620] Acquiring the partition lock for write op
0812 06:37:19.412825 (+     6us) write_op.cc:641] Partition lock acquired for write op
0812 06:37:19.412827 (+     2us) tablet.cc:685] Acquiring locks for 10 operations
0812 06:37:19.412882 (+    55us) tablet.cc:701] Row locks acquired
0812 06:37:19.412883 (+     1us) write_op.cc:260] PREPARE: finished
0812 06:37:19.412907 (+    24us) write_op.cc:270] Start()
0812 06:37:19.412938 (+    31us) write_op.cc:276] Timestamp: P: 1786516639412833 usec, L: 0
0812 06:37:19.412939 (+     1us) op_driver.cc:348] REPLICATION: starting
0812 06:37:19.413026 (+    87us) log.cc:844] Serialized 696 byte log entry
0812 06:37:21.183619 (+1770593us) op_driver.cc:464] REPLICATION: finished
0812 06:37:21.183702 (+    83us) write_op.cc:301] APPLY: starting
0812 06:37:21.183716 (+    14us) tablet.cc:1367] starting BulkCheckPresence
0812 06:37:21.183752 (+    36us) tablet.cc:1370] finished BulkCheckPresence
0812 06:37:21.183767 (+    15us) tablet.cc:1372] starting ApplyRowOperation cycle
0812 06:37:21.183963 (+   196us) tablet.cc:1383] finished ApplyRowOperation cycle
0812 06:37:21.183993 (+    30us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0812 06:37:21.183998 (+     5us) write_op.cc:312] APPLY: finished
0812 06:37:21.184053 (+    55us) log.cc:844] Serialized 77 byte log entry
0812 06:37:21.184091 (+    38us) write_op.cc:489] Releasing partition, row and schema locks
0812 06:37:21.184106 (+    15us) write_op.cc:454] Released schema lock
0812 06:37:21.184112 (+     6us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":32,"num_ops":10,"prepare.queue_time_us":4666,"prepare.run_cpu_time_us":431,"prepare.run_wall_time_us":478,"raft.queue_time_us":33,"raft.run_cpu_time_us":4,"raft.run_wall_time_us":5,"replication_time_us":1770652}]]}
I20260812 06:37:21.192752 11550 mvcc.cc:204] Tried to move back new op lower bound from 7317572162192199680 to 7317572161249247232. Current Snapshot: MvccSnapshot[applied={T|T < 7317572162192199680 or (T in {7317572162192199680})}]
I20260812 06:37:21.194379 11570 mvcc.cc:204] Tried to move back new op lower bound from 7317572162192199680 to 7317572161249247232. Current Snapshot: MvccSnapshot[applied={T|T < 7317572155017572352}]
I20260812 06:37:22.269325 11178 tablet_bootstrap.cc:492] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e: Bootstrap replayed 1/1 log segments. Stats: ops{read=3654 overwritten=27 applied=3619 ignored=0} inserts{seen=36010 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 8 replicates
I20260812 06:37:22.269905 11178 tablet_bootstrap.cc:492] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e: Bootstrap complete.
I20260812 06:37:22.620656 11178 ts_tablet_manager.cc:1403] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e: Time spent bootstrapping tablet: real 3.997s	user 2.866s	sys 0.237s
I20260812 06:37:22.622936 11178 raft_consensus.cc:359] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [term 23 FOLLOWER]: Replica starting. Triggering 8 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df4308c852240859ada53d216ab69ae" member_type: VOTER last_known_addr { host: "127.4.177.6" port: 34269 } } peers { permanent_uuid: "42895a9bcba64aa484f6509605035d02" member_type: VOTER last_known_addr { host: "127.4.177.4" port: 38181 } } peers { permanent_uuid: "222b20de5a1145c6b8a11df8374e8790" member_type: VOTER last_known_addr { host: "127.4.177.5" port: 43521 } } peers { permanent_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 40897 } } peers { permanent_uuid: "8d90d61da62549208fe117b4384a469b" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 41539 } } peers { permanent_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" member_type: VOTER last_known_addr { host: "127.4.177.7" port: 46299 } } peers { permanent_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 41057 } }
I20260812 06:37:22.625785 11178 raft_consensus.cc:740] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [term 23 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7962f587ccd842ad8a3f6ed9a5d7707e, State: Initialized, Role: FOLLOWER
I20260812 06:37:22.626755 11178 consensus_queue.cc:260] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3619, Last appended: 23.3627, Last appended by leader: 3627, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df4308c852240859ada53d216ab69ae" member_type: VOTER last_known_addr { host: "127.4.177.6" port: 34269 } } peers { permanent_uuid: "42895a9bcba64aa484f6509605035d02" member_type: VOTER last_known_addr { host: "127.4.177.4" port: 38181 } } peers { permanent_uuid: "222b20de5a1145c6b8a11df8374e8790" member_type: VOTER last_known_addr { host: "127.4.177.5" port: 43521 } } peers { permanent_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 40897 } } peers { permanent_uuid: "8d90d61da62549208fe117b4384a469b" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 41539 } } peers { permanent_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" member_type: VOTER last_known_addr { host: "127.4.177.7" port: 46299 } } peers { permanent_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 41057 } }
I20260812 06:37:22.628345 11178 ts_tablet_manager.cc:1434] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260812 06:37:23.657677 11614 raft_consensus.cc:493] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 2df4308c852240859ada53d216ab69ae)
I20260812 06:37:23.657871 11614 raft_consensus.cc:515] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df4308c852240859ada53d216ab69ae" member_type: VOTER last_known_addr { host: "127.4.177.6" port: 34269 } } peers { permanent_uuid: "42895a9bcba64aa484f6509605035d02" member_type: VOTER last_known_addr { host: "127.4.177.4" port: 38181 } } peers { permanent_uuid: "222b20de5a1145c6b8a11df8374e8790" member_type: VOTER last_known_addr { host: "127.4.177.5" port: 43521 } } peers { permanent_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 40897 } } peers { permanent_uuid: "8d90d61da62549208fe117b4384a469b" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 41539 } } peers { permanent_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" member_type: VOTER last_known_addr { host: "127.4.177.7" port: 46299 } } peers { permanent_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 41057 } }
I20260812 06:37:23.658715 11614 leader_election.cc:290] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 2df4308c852240859ada53d216ab69ae (127.4.177.6:34269), 42895a9bcba64aa484f6509605035d02 (127.4.177.4:38181), 222b20de5a1145c6b8a11df8374e8790 (127.4.177.5:43521), 7962f587ccd842ad8a3f6ed9a5d7707e (127.4.177.3:40897), b037d17b21f8422fa5e41b70eb4dcd08 (127.4.177.7:46299), ec6c8d818e744cdbb6b0d15b321010e0 (127.4.177.1:41057)
I20260812 06:37:23.659240  6700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "8d90d61da62549208fe117b4384a469b" candidate_term: 26 candidate_status { last_received { term: 25 index: 3948 } } ignore_live_leader: false dest_uuid: "42895a9bcba64aa484f6509605035d02" is_pre_election: true
I20260812 06:37:23.659240  9425 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "8d90d61da62549208fe117b4384a469b" candidate_term: 26 candidate_status { last_received { term: 25 index: 3948 } } ignore_live_leader: false dest_uuid: "222b20de5a1145c6b8a11df8374e8790" is_pre_election: true
I20260812 06:37:23.659214  7318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "8d90d61da62549208fe117b4384a469b" candidate_term: 26 candidate_status { last_received { term: 25 index: 3948 } } ignore_live_leader: false dest_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" is_pre_election: true
I20260812 06:37:23.659392 11291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "8d90d61da62549208fe117b4384a469b" candidate_term: 26 candidate_status { last_received { term: 25 index: 3948 } } ignore_live_leader: false dest_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" is_pre_election: true
I20260812 06:37:23.659672 11291 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8d90d61da62549208fe117b4384a469b in term 23.
I20260812 06:37:23.659899  6046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "8d90d61da62549208fe117b4384a469b" candidate_term: 26 candidate_status { last_received { term: 25 index: 3948 } } ignore_live_leader: false dest_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" is_pre_election: true
I20260812 06:37:23.660327  6150 leader_election.cc:304] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 6 responses out of 7 voters: 2 yes votes; 4 no votes. yes voters: 7962f587ccd842ad8a3f6ed9a5d7707e, 8d90d61da62549208fe117b4384a469b; no voters: 222b20de5a1145c6b8a11df8374e8790, 42895a9bcba64aa484f6509605035d02, b037d17b21f8422fa5e41b70eb4dcd08, ec6c8d818e744cdbb6b0d15b321010e0
I20260812 06:37:23.660514 11614 raft_consensus.cc:2749] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260812 06:37:23.667738 11615 raft_consensus.cc:493] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 2df4308c852240859ada53d216ab69ae)
I20260812 06:37:23.667893 11615 raft_consensus.cc:515] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df4308c852240859ada53d216ab69ae" member_type: VOTER last_known_addr { host: "127.4.177.6" port: 34269 } } peers { permanent_uuid: "42895a9bcba64aa484f6509605035d02" member_type: VOTER last_known_addr { host: "127.4.177.4" port: 38181 } } peers { permanent_uuid: "222b20de5a1145c6b8a11df8374e8790" member_type: VOTER last_known_addr { host: "127.4.177.5" port: 43521 } } peers { permanent_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 40897 } } peers { permanent_uuid: "8d90d61da62549208fe117b4384a469b" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 41539 } } peers { permanent_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" member_type: VOTER last_known_addr { host: "127.4.177.7" port: 46299 } } peers { permanent_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 41057 } }
I20260812 06:37:23.667963 11620 raft_consensus.cc:493] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 2df4308c852240859ada53d216ab69ae)
I20260812 06:37:23.668084 11620 raft_consensus.cc:515] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df4308c852240859ada53d216ab69ae" member_type: VOTER last_known_addr { host: "127.4.177.6" port: 34269 } } peers { permanent_uuid: "42895a9bcba64aa484f6509605035d02" member_type: VOTER last_known_addr { host: "127.4.177.4" port: 38181 } } peers { permanent_uuid: "222b20de5a1145c6b8a11df8374e8790" member_type: VOTER last_known_addr { host: "127.4.177.5" port: 43521 } } peers { permanent_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 40897 } } peers { permanent_uuid: "8d90d61da62549208fe117b4384a469b" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 41539 } } peers { permanent_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" member_type: VOTER last_known_addr { host: "127.4.177.7" port: 46299 } } peers { permanent_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 41057 } }
I20260812 06:37:23.668803 11620 leader_election.cc:290] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 2df4308c852240859ada53d216ab69ae (127.4.177.6:34269), 222b20de5a1145c6b8a11df8374e8790 (127.4.177.5:43521), 7962f587ccd842ad8a3f6ed9a5d7707e (127.4.177.3:40897), 8d90d61da62549208fe117b4384a469b (127.4.177.2:41539), b037d17b21f8422fa5e41b70eb4dcd08 (127.4.177.7:46299), ec6c8d818e744cdbb6b0d15b321010e0 (127.4.177.1:41057)
I20260812 06:37:23.669153  9425 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "42895a9bcba64aa484f6509605035d02" candidate_term: 26 candidate_status { last_received { term: 25 index: 3953 } } ignore_live_leader: false dest_uuid: "222b20de5a1145c6b8a11df8374e8790" is_pre_election: true
I20260812 06:37:23.669283  9425 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 42895a9bcba64aa484f6509605035d02 in term 25.
I20260812 06:37:23.669291  7318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "222b20de5a1145c6b8a11df8374e8790" candidate_term: 26 candidate_status { last_received { term: 25 index: 3953 } } ignore_live_leader: false dest_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" is_pre_election: true
I20260812 06:37:23.669360 11293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "42895a9bcba64aa484f6509605035d02" candidate_term: 26 candidate_status { last_received { term: 25 index: 3953 } } ignore_live_leader: false dest_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" is_pre_election: true
I20260812 06:37:23.669476  6700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "222b20de5a1145c6b8a11df8374e8790" candidate_term: 26 candidate_status { last_received { term: 25 index: 3953 } } ignore_live_leader: false dest_uuid: "42895a9bcba64aa484f6509605035d02" is_pre_election: true
I20260812 06:37:23.669482 11293 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 42895a9bcba64aa484f6509605035d02 in term 23.
I20260812 06:37:23.669595  6700 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 222b20de5a1145c6b8a11df8374e8790 in term 25.
I20260812 06:37:23.669626  6238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "222b20de5a1145c6b8a11df8374e8790" candidate_term: 26 candidate_status { last_received { term: 25 index: 3953 } } ignore_live_leader: false dest_uuid: "8d90d61da62549208fe117b4384a469b" is_pre_election: true
I20260812 06:37:23.669757  6238 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 222b20de5a1145c6b8a11df8374e8790 in term 25.
I20260812 06:37:23.669876 11615 leader_election.cc:290] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 2df4308c852240859ada53d216ab69ae (127.4.177.6:34269), 42895a9bcba64aa484f6509605035d02 (127.4.177.4:38181), 7962f587ccd842ad8a3f6ed9a5d7707e (127.4.177.3:40897), 8d90d61da62549208fe117b4384a469b (127.4.177.2:41539), b037d17b21f8422fa5e41b70eb4dcd08 (127.4.177.7:46299), ec6c8d818e744cdbb6b0d15b321010e0 (127.4.177.1:41057)
I20260812 06:37:23.669984  6236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "42895a9bcba64aa484f6509605035d02" candidate_term: 26 candidate_status { last_received { term: 25 index: 3953 } } ignore_live_leader: false dest_uuid: "8d90d61da62549208fe117b4384a469b" is_pre_election: true
I20260812 06:37:23.670068 11291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "222b20de5a1145c6b8a11df8374e8790" candidate_term: 26 candidate_status { last_received { term: 25 index: 3953 } } ignore_live_leader: false dest_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" is_pre_election: true
I20260812 06:37:23.670111  6236 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 42895a9bcba64aa484f6509605035d02 in term 25.
I20260812 06:37:23.670171 11291 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 222b20de5a1145c6b8a11df8374e8790 in term 23.
I20260812 06:37:23.670394  6046 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "222b20de5a1145c6b8a11df8374e8790" candidate_term: 26 candidate_status { last_received { term: 25 index: 3953 } } ignore_live_leader: false dest_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" is_pre_election: true
I20260812 06:37:23.670418  6597 leader_election.cc:304] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 7 voters: 4 yes votes; 0 no votes. yes voters: 222b20de5a1145c6b8a11df8374e8790, 42895a9bcba64aa484f6509605035d02, 7962f587ccd842ad8a3f6ed9a5d7707e, 8d90d61da62549208fe117b4384a469b; no voters: 
I20260812 06:37:23.670504  6048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "42895a9bcba64aa484f6509605035d02" candidate_term: 26 candidate_status { last_received { term: 25 index: 3953 } } ignore_live_leader: false dest_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" is_pre_election: true
I20260812 06:37:23.670603  7318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "42895a9bcba64aa484f6509605035d02" candidate_term: 26 candidate_status { last_received { term: 25 index: 3953 } } ignore_live_leader: false dest_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" is_pre_election: true
I20260812 06:37:23.670644 11620 raft_consensus.cc:2804] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260812 06:37:23.670691 11620 raft_consensus.cc:493] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [term 25 FOLLOWER]: Starting leader election (detected failure of leader 2df4308c852240859ada53d216ab69ae)
I20260812 06:37:23.670729 11620 raft_consensus.cc:3060] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [term 25 FOLLOWER]: Advancing to term 26
I20260812 06:37:23.670812  9315 leader_election.cc:304] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 5 responses out of 7 voters: 4 yes votes; 1 no votes. yes voters: 222b20de5a1145c6b8a11df8374e8790, 42895a9bcba64aa484f6509605035d02, 7962f587ccd842ad8a3f6ed9a5d7707e, 8d90d61da62549208fe117b4384a469b; no voters: b037d17b21f8422fa5e41b70eb4dcd08
I20260812 06:37:23.671025 11615 raft_consensus.cc:2804] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260812 06:37:23.671097 11615 raft_consensus.cc:493] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [term 25 FOLLOWER]: Starting leader election (detected failure of leader 2df4308c852240859ada53d216ab69ae)
I20260812 06:37:23.671154 11615 raft_consensus.cc:3060] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [term 25 FOLLOWER]: Advancing to term 26
I20260812 06:37:23.672103 11620 raft_consensus.cc:515] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df4308c852240859ada53d216ab69ae" member_type: VOTER last_known_addr { host: "127.4.177.6" port: 34269 } } peers { permanent_uuid: "42895a9bcba64aa484f6509605035d02" member_type: VOTER last_known_addr { host: "127.4.177.4" port: 38181 } } peers { permanent_uuid: "222b20de5a1145c6b8a11df8374e8790" member_type: VOTER last_known_addr { host: "127.4.177.5" port: 43521 } } peers { permanent_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 40897 } } peers { permanent_uuid: "8d90d61da62549208fe117b4384a469b" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 41539 } } peers { permanent_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" member_type: VOTER last_known_addr { host: "127.4.177.7" port: 46299 } } peers { permanent_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 41057 } }
I20260812 06:37:23.672168 11615 raft_consensus.cc:515] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df4308c852240859ada53d216ab69ae" member_type: VOTER last_known_addr { host: "127.4.177.6" port: 34269 } } peers { permanent_uuid: "42895a9bcba64aa484f6509605035d02" member_type: VOTER last_known_addr { host: "127.4.177.4" port: 38181 } } peers { permanent_uuid: "222b20de5a1145c6b8a11df8374e8790" member_type: VOTER last_known_addr { host: "127.4.177.5" port: 43521 } } peers { permanent_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 40897 } } peers { permanent_uuid: "8d90d61da62549208fe117b4384a469b" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 41539 } } peers { permanent_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" member_type: VOTER last_known_addr { host: "127.4.177.7" port: 46299 } } peers { permanent_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 41057 } }
I20260812 06:37:23.673041 11620 leader_election.cc:290] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [CANDIDATE]: Term 26 election: Requested vote from peers 2df4308c852240859ada53d216ab69ae (127.4.177.6:34269), 222b20de5a1145c6b8a11df8374e8790 (127.4.177.5:43521), 7962f587ccd842ad8a3f6ed9a5d7707e (127.4.177.3:40897), 8d90d61da62549208fe117b4384a469b (127.4.177.2:41539), b037d17b21f8422fa5e41b70eb4dcd08 (127.4.177.7:46299), ec6c8d818e744cdbb6b0d15b321010e0 (127.4.177.1:41057)
I20260812 06:37:23.673032  7318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "222b20de5a1145c6b8a11df8374e8790" candidate_term: 26 candidate_status { last_received { term: 25 index: 3953 } } ignore_live_leader: false dest_uuid: "b037d17b21f8422fa5e41b70eb4dcd08"
I20260812 06:37:23.673086  9425 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "42895a9bcba64aa484f6509605035d02" candidate_term: 26 candidate_status { last_received { term: 25 index: 3953 } } ignore_live_leader: false dest_uuid: "222b20de5a1145c6b8a11df8374e8790"
I20260812 06:37:23.673164  6238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "42895a9bcba64aa484f6509605035d02" candidate_term: 26 candidate_status { last_received { term: 25 index: 3953 } } ignore_live_leader: false dest_uuid: "8d90d61da62549208fe117b4384a469b"
I20260812 06:37:23.673220  9425 raft_consensus.cc:2393] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate 42895a9bcba64aa484f6509605035d02 in current term 26: Already voted for candidate 222b20de5a1145c6b8a11df8374e8790 in this term.
I20260812 06:37:23.673257  6238 raft_consensus.cc:3060] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b [term 25 FOLLOWER]: Advancing to term 26
I20260812 06:37:23.673369 11615 leader_election.cc:290] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [CANDIDATE]: Term 26 election: Requested vote from peers 2df4308c852240859ada53d216ab69ae (127.4.177.6:34269), 42895a9bcba64aa484f6509605035d02 (127.4.177.4:38181), 7962f587ccd842ad8a3f6ed9a5d7707e (127.4.177.3:40897), 8d90d61da62549208fe117b4384a469b (127.4.177.2:41539), b037d17b21f8422fa5e41b70eb4dcd08 (127.4.177.7:46299), ec6c8d818e744cdbb6b0d15b321010e0 (127.4.177.1:41057)
I20260812 06:37:23.673627 11293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "222b20de5a1145c6b8a11df8374e8790" candidate_term: 26 candidate_status { last_received { term: 25 index: 3953 } } ignore_live_leader: false dest_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e"
I20260812 06:37:23.673753 11293 raft_consensus.cc:3060] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [term 23 FOLLOWER]: Advancing to term 26
I20260812 06:37:23.673974  6048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "222b20de5a1145c6b8a11df8374e8790" candidate_term: 26 candidate_status { last_received { term: 25 index: 3953 } } ignore_live_leader: false dest_uuid: "ec6c8d818e744cdbb6b0d15b321010e0"
I20260812 06:37:23.673264  6236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "222b20de5a1145c6b8a11df8374e8790" candidate_term: 26 candidate_status { last_received { term: 25 index: 3953 } } ignore_live_leader: false dest_uuid: "8d90d61da62549208fe117b4384a469b"
I20260812 06:37:23.674438  6238 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 42895a9bcba64aa484f6509605035d02 in term 26.
I20260812 06:37:23.674438 11291 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "42895a9bcba64aa484f6509605035d02" candidate_term: 26 candidate_status { last_received { term: 25 index: 3953 } } ignore_live_leader: false dest_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e"
I20260812 06:37:23.674480  6700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "222b20de5a1145c6b8a11df8374e8790" candidate_term: 26 candidate_status { last_received { term: 25 index: 3953 } } ignore_live_leader: false dest_uuid: "42895a9bcba64aa484f6509605035d02"
I20260812 06:37:23.674600  6700 raft_consensus.cc:2393] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate 222b20de5a1145c6b8a11df8374e8790 in current term 26: Already voted for candidate 42895a9bcba64aa484f6509605035d02 in this term.
I20260812 06:37:23.675138  7318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "42895a9bcba64aa484f6509605035d02" candidate_term: 26 candidate_status { last_received { term: 25 index: 3953 } } ignore_live_leader: false dest_uuid: "b037d17b21f8422fa5e41b70eb4dcd08"
I20260812 06:37:23.675217  9315 leader_election.cc:304] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 5 responses out of 7 voters: 1 yes votes; 4 no votes. yes voters: 222b20de5a1145c6b8a11df8374e8790; no voters: 42895a9bcba64aa484f6509605035d02, 8d90d61da62549208fe117b4384a469b, b037d17b21f8422fa5e41b70eb4dcd08, ec6c8d818e744cdbb6b0d15b321010e0
I20260812 06:37:23.675326  6048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "42895a9bcba64aa484f6509605035d02" candidate_term: 26 candidate_status { last_received { term: 25 index: 3953 } } ignore_live_leader: false dest_uuid: "ec6c8d818e744cdbb6b0d15b321010e0"
I20260812 06:37:23.675377 11615 raft_consensus.cc:2749] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [term 26 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
I20260812 06:37:23.675751  6595 leader_election.cc:304] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 6 responses out of 7 voters: 2 yes votes; 4 no votes. yes voters: 42895a9bcba64aa484f6509605035d02, 8d90d61da62549208fe117b4384a469b; no voters: 222b20de5a1145c6b8a11df8374e8790, 7962f587ccd842ad8a3f6ed9a5d7707e, b037d17b21f8422fa5e41b70eb4dcd08, ec6c8d818e744cdbb6b0d15b321010e0
I20260812 06:37:23.675952 11620 raft_consensus.cc:2749] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [term 26 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
I20260812 06:37:23.676029 11293 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 222b20de5a1145c6b8a11df8374e8790 in term 26.
I20260812 06:37:24.118690 11622 raft_consensus.cc:493] T 7065bb403e3047dd9a0853c49318ef1f P ec6c8d818e744cdbb6b0d15b321010e0 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 2df4308c852240859ada53d216ab69ae)
I20260812 06:37:24.118867 11622 raft_consensus.cc:515] T 7065bb403e3047dd9a0853c49318ef1f P ec6c8d818e744cdbb6b0d15b321010e0 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df4308c852240859ada53d216ab69ae" member_type: VOTER last_known_addr { host: "127.4.177.6" port: 34269 } } peers { permanent_uuid: "42895a9bcba64aa484f6509605035d02" member_type: VOTER last_known_addr { host: "127.4.177.4" port: 38181 } } peers { permanent_uuid: "222b20de5a1145c6b8a11df8374e8790" member_type: VOTER last_known_addr { host: "127.4.177.5" port: 43521 } } peers { permanent_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 40897 } } peers { permanent_uuid: "8d90d61da62549208fe117b4384a469b" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 41539 } } peers { permanent_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" member_type: VOTER last_known_addr { host: "127.4.177.7" port: 46299 } } peers { permanent_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 41057 } }
I20260812 06:37:24.119853 11622 leader_election.cc:290] T 7065bb403e3047dd9a0853c49318ef1f P ec6c8d818e744cdbb6b0d15b321010e0 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 2df4308c852240859ada53d216ab69ae (127.4.177.6:34269), 42895a9bcba64aa484f6509605035d02 (127.4.177.4:38181), 222b20de5a1145c6b8a11df8374e8790 (127.4.177.5:43521), 7962f587ccd842ad8a3f6ed9a5d7707e (127.4.177.3:40897), 8d90d61da62549208fe117b4384a469b (127.4.177.2:41539), b037d17b21f8422fa5e41b70eb4dcd08 (127.4.177.7:46299)
I20260812 06:37:24.120136 11623 raft_consensus.cc:493] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 2df4308c852240859ada53d216ab69ae)
I20260812 06:37:24.120299 11623 raft_consensus.cc:515] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df4308c852240859ada53d216ab69ae" member_type: VOTER last_known_addr { host: "127.4.177.6" port: 34269 } } peers { permanent_uuid: "42895a9bcba64aa484f6509605035d02" member_type: VOTER last_known_addr { host: "127.4.177.4" port: 38181 } } peers { permanent_uuid: "222b20de5a1145c6b8a11df8374e8790" member_type: VOTER last_known_addr { host: "127.4.177.5" port: 43521 } } peers { permanent_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 40897 } } peers { permanent_uuid: "8d90d61da62549208fe117b4384a469b" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 41539 } } peers { permanent_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" member_type: VOTER last_known_addr { host: "127.4.177.7" port: 46299 } } peers { permanent_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 41057 } }
I20260812 06:37:24.120497  9425 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" candidate_term: 26 candidate_status { last_received { term: 25 index: 3891 } } ignore_live_leader: false dest_uuid: "222b20de5a1145c6b8a11df8374e8790" is_pre_election: true
I20260812 06:37:24.120682  9425 raft_consensus.cc:2393] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ec6c8d818e744cdbb6b0d15b321010e0 in current term 26: Already voted for candidate 222b20de5a1145c6b8a11df8374e8790 in this term.
I20260812 06:37:24.120663  6238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" candidate_term: 26 candidate_status { last_received { term: 25 index: 3891 } } ignore_live_leader: false dest_uuid: "8d90d61da62549208fe117b4384a469b" is_pre_election: true
I20260812 06:37:24.120816  6238 raft_consensus.cc:2393] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ec6c8d818e744cdbb6b0d15b321010e0 in current term 26: Already voted for candidate 42895a9bcba64aa484f6509605035d02 in this term.
I20260812 06:37:24.121289  7318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" candidate_term: 26 candidate_status { last_received { term: 25 index: 3891 } } ignore_live_leader: false dest_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" is_pre_election: true
I20260812 06:37:24.121461  7318 raft_consensus.cc:2410] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ec6c8d818e744cdbb6b0d15b321010e0 for term 26 because replica has last-logged OpId of term: 25 index: 3918, which is greater than that of the candidate, which has last-logged OpId of term: 25 index: 3891.
I20260812 06:37:24.121515  6700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" candidate_term: 26 candidate_status { last_received { term: 25 index: 3891 } } ignore_live_leader: false dest_uuid: "42895a9bcba64aa484f6509605035d02" is_pre_election: true
I20260812 06:37:24.121641 11623 leader_election.cc:290] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 2df4308c852240859ada53d216ab69ae (127.4.177.6:34269), 42895a9bcba64aa484f6509605035d02 (127.4.177.4:38181), 222b20de5a1145c6b8a11df8374e8790 (127.4.177.5:43521), 7962f587ccd842ad8a3f6ed9a5d7707e (127.4.177.3:40897), 8d90d61da62549208fe117b4384a469b (127.4.177.2:41539), ec6c8d818e744cdbb6b0d15b321010e0 (127.4.177.1:41057)
I20260812 06:37:24.121677  6700 raft_consensus.cc:2393] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ec6c8d818e744cdbb6b0d15b321010e0 in current term 26: Already voted for candidate 42895a9bcba64aa484f6509605035d02 in this term.
I20260812 06:37:24.122174 11293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" candidate_term: 26 candidate_status { last_received { term: 25 index: 3918 } } ignore_live_leader: false dest_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" is_pre_election: true
I20260812 06:37:24.122290  6238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" candidate_term: 26 candidate_status { last_received { term: 25 index: 3918 } } ignore_live_leader: false dest_uuid: "8d90d61da62549208fe117b4384a469b" is_pre_election: true
I20260812 06:37:24.122392 11293 raft_consensus.cc:2393] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b037d17b21f8422fa5e41b70eb4dcd08 in current term 26: Already voted for candidate 222b20de5a1145c6b8a11df8374e8790 in this term.
I20260812 06:37:24.122424  6238 raft_consensus.cc:2393] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b037d17b21f8422fa5e41b70eb4dcd08 in current term 26: Already voted for candidate 42895a9bcba64aa484f6509605035d02 in this term.
I20260812 06:37:24.122686  6048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" candidate_term: 26 candidate_status { last_received { term: 25 index: 3918 } } ignore_live_leader: false dest_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" is_pre_election: true
I20260812 06:37:24.122718  9425 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" candidate_term: 26 candidate_status { last_received { term: 25 index: 3918 } } ignore_live_leader: false dest_uuid: "222b20de5a1145c6b8a11df8374e8790" is_pre_election: true
I20260812 06:37:24.122804  6048 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P ec6c8d818e744cdbb6b0d15b321010e0 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b037d17b21f8422fa5e41b70eb4dcd08 in term 25.
I20260812 06:37:24.122849  9425 raft_consensus.cc:2393] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b037d17b21f8422fa5e41b70eb4dcd08 in current term 26: Already voted for candidate 222b20de5a1145c6b8a11df8374e8790 in this term.
I20260812 06:37:24.123018  6700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" candidate_term: 26 candidate_status { last_received { term: 25 index: 3918 } } ignore_live_leader: false dest_uuid: "42895a9bcba64aa484f6509605035d02" is_pre_election: true
I20260812 06:37:24.123195  6700 raft_consensus.cc:2393] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b037d17b21f8422fa5e41b70eb4dcd08 in current term 26: Already voted for candidate 42895a9bcba64aa484f6509605035d02 in this term.
I20260812 06:37:24.123276  5945 leader_election.cc:304] T 7065bb403e3047dd9a0853c49318ef1f P ec6c8d818e744cdbb6b0d15b321010e0 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 5 responses out of 7 voters: 1 yes votes; 4 no votes. yes voters: ec6c8d818e744cdbb6b0d15b321010e0; no voters: 222b20de5a1145c6b8a11df8374e8790, 42895a9bcba64aa484f6509605035d02, 8d90d61da62549208fe117b4384a469b, b037d17b21f8422fa5e41b70eb4dcd08
I20260812 06:37:24.123497 11622 raft_consensus.cc:3060] T 7065bb403e3047dd9a0853c49318ef1f P ec6c8d818e744cdbb6b0d15b321010e0 [term 25 FOLLOWER]: Advancing to term 26
I20260812 06:37:24.123533  7211 leader_election.cc:304] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 6 responses out of 7 voters: 2 yes votes; 4 no votes. yes voters: b037d17b21f8422fa5e41b70eb4dcd08, ec6c8d818e744cdbb6b0d15b321010e0; no voters: 222b20de5a1145c6b8a11df8374e8790, 42895a9bcba64aa484f6509605035d02, 7962f587ccd842ad8a3f6ed9a5d7707e, 8d90d61da62549208fe117b4384a469b
I20260812 06:37:24.123729 11623 raft_consensus.cc:3060] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08 [term 25 FOLLOWER]: Advancing to term 26
I20260812 06:37:24.124696 11622 raft_consensus.cc:2749] T 7065bb403e3047dd9a0853c49318ef1f P ec6c8d818e744cdbb6b0d15b321010e0 [term 26 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260812 06:37:24.125008 11623 raft_consensus.cc:2749] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08 [term 26 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260812 06:37:24.125516 11293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" candidate_term: 26 candidate_status { last_received { term: 25 index: 3891 } } ignore_live_leader: false dest_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" is_pre_election: true
I20260812 06:37:24.125702 11293 raft_consensus.cc:2393] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ec6c8d818e744cdbb6b0d15b321010e0 in current term 26: Already voted for candidate 222b20de5a1145c6b8a11df8374e8790 in this term.
W20260812 06:37:25.175614  6150 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.4.177.6:34269 timed out after 1.515s (SENT) [suppressed 1 similar messages]
W20260812 06:37:25.195820  6597 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.4.177.6:34269 timed out after 1.526s (SENT) [suppressed 1 similar messages]
W20260812 06:37:25.365412  9315 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.4.177.6:34269 timed out after 1.695s (SENT) [suppressed 2 similar messages]
I20260812 06:37:25.534847 11635 raft_consensus.cc:493] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 06:37:25.535048 11635 raft_consensus.cc:515] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df4308c852240859ada53d216ab69ae" member_type: VOTER last_known_addr { host: "127.4.177.6" port: 34269 } } peers { permanent_uuid: "42895a9bcba64aa484f6509605035d02" member_type: VOTER last_known_addr { host: "127.4.177.4" port: 38181 } } peers { permanent_uuid: "222b20de5a1145c6b8a11df8374e8790" member_type: VOTER last_known_addr { host: "127.4.177.5" port: 43521 } } peers { permanent_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 40897 } } peers { permanent_uuid: "8d90d61da62549208fe117b4384a469b" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 41539 } } peers { permanent_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" member_type: VOTER last_known_addr { host: "127.4.177.7" port: 46299 } } peers { permanent_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 41057 } }
I20260812 06:37:25.535696 11635 leader_election.cc:290] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 2df4308c852240859ada53d216ab69ae (127.4.177.6:34269), 42895a9bcba64aa484f6509605035d02 (127.4.177.4:38181), 7962f587ccd842ad8a3f6ed9a5d7707e (127.4.177.3:40897), 8d90d61da62549208fe117b4384a469b (127.4.177.2:41539), b037d17b21f8422fa5e41b70eb4dcd08 (127.4.177.7:46299), ec6c8d818e744cdbb6b0d15b321010e0 (127.4.177.1:41057)
I20260812 06:37:25.536221  7318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "222b20de5a1145c6b8a11df8374e8790" candidate_term: 27 candidate_status { last_received { term: 25 index: 3953 } } ignore_live_leader: false dest_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" is_pre_election: true
I20260812 06:37:25.536226 11293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "222b20de5a1145c6b8a11df8374e8790" candidate_term: 27 candidate_status { last_received { term: 25 index: 3953 } } ignore_live_leader: false dest_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" is_pre_election: true
I20260812 06:37:25.536230  6700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "222b20de5a1145c6b8a11df8374e8790" candidate_term: 27 candidate_status { last_received { term: 25 index: 3953 } } ignore_live_leader: false dest_uuid: "42895a9bcba64aa484f6509605035d02" is_pre_election: true
I20260812 06:37:25.536373 11293 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 222b20de5a1145c6b8a11df8374e8790 in term 26.
I20260812 06:37:25.536410  7318 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 222b20de5a1145c6b8a11df8374e8790 in term 26.
I20260812 06:37:25.536420  6700 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 222b20de5a1145c6b8a11df8374e8790 in term 26.
I20260812 06:37:25.536631  6048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "222b20de5a1145c6b8a11df8374e8790" candidate_term: 27 candidate_status { last_received { term: 25 index: 3953 } } ignore_live_leader: false dest_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" is_pre_election: true
I20260812 06:37:25.536688  6238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "222b20de5a1145c6b8a11df8374e8790" candidate_term: 27 candidate_status { last_received { term: 25 index: 3953 } } ignore_live_leader: false dest_uuid: "8d90d61da62549208fe117b4384a469b" is_pre_election: true
I20260812 06:37:25.536795  6048 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P ec6c8d818e744cdbb6b0d15b321010e0 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 222b20de5a1145c6b8a11df8374e8790 in term 26.
I20260812 06:37:25.536825  6238 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 222b20de5a1145c6b8a11df8374e8790 in term 26.
I20260812 06:37:25.537153  9315 leader_election.cc:304] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 7 voters: 4 yes votes; 0 no votes. yes voters: 222b20de5a1145c6b8a11df8374e8790, 42895a9bcba64aa484f6509605035d02, 7962f587ccd842ad8a3f6ed9a5d7707e, b037d17b21f8422fa5e41b70eb4dcd08; no voters: 
I20260812 06:37:25.537367 11635 raft_consensus.cc:2804] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260812 06:37:25.537453 11635 raft_consensus.cc:493] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 06:37:25.537524 11635 raft_consensus.cc:3060] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [term 26 FOLLOWER]: Advancing to term 27
I20260812 06:37:25.538524 11635 raft_consensus.cc:515] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df4308c852240859ada53d216ab69ae" member_type: VOTER last_known_addr { host: "127.4.177.6" port: 34269 } } peers { permanent_uuid: "42895a9bcba64aa484f6509605035d02" member_type: VOTER last_known_addr { host: "127.4.177.4" port: 38181 } } peers { permanent_uuid: "222b20de5a1145c6b8a11df8374e8790" member_type: VOTER last_known_addr { host: "127.4.177.5" port: 43521 } } peers { permanent_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 40897 } } peers { permanent_uuid: "8d90d61da62549208fe117b4384a469b" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 41539 } } peers { permanent_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" member_type: VOTER last_known_addr { host: "127.4.177.7" port: 46299 } } peers { permanent_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 41057 } }
I20260812 06:37:25.539227 11635 leader_election.cc:290] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [CANDIDATE]: Term 27 election: Requested vote from peers 2df4308c852240859ada53d216ab69ae (127.4.177.6:34269), 42895a9bcba64aa484f6509605035d02 (127.4.177.4:38181), 7962f587ccd842ad8a3f6ed9a5d7707e (127.4.177.3:40897), 8d90d61da62549208fe117b4384a469b (127.4.177.2:41539), b037d17b21f8422fa5e41b70eb4dcd08 (127.4.177.7:46299), ec6c8d818e744cdbb6b0d15b321010e0 (127.4.177.1:41057)
I20260812 06:37:25.539484  6700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "222b20de5a1145c6b8a11df8374e8790" candidate_term: 27 candidate_status { last_received { term: 25 index: 3953 } } ignore_live_leader: false dest_uuid: "42895a9bcba64aa484f6509605035d02"
I20260812 06:37:25.539613  6700 raft_consensus.cc:3060] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [term 26 FOLLOWER]: Advancing to term 27
I20260812 06:37:25.539602  7318 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "222b20de5a1145c6b8a11df8374e8790" candidate_term: 27 candidate_status { last_received { term: 25 index: 3953 } } ignore_live_leader: false dest_uuid: "b037d17b21f8422fa5e41b70eb4dcd08"
I20260812 06:37:25.539716  7318 raft_consensus.cc:3060] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08 [term 26 FOLLOWER]: Advancing to term 27
I20260812 06:37:25.539726 11293 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "222b20de5a1145c6b8a11df8374e8790" candidate_term: 27 candidate_status { last_received { term: 25 index: 3953 } } ignore_live_leader: false dest_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e"
I20260812 06:37:25.539726  6238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "222b20de5a1145c6b8a11df8374e8790" candidate_term: 27 candidate_status { last_received { term: 25 index: 3953 } } ignore_live_leader: false dest_uuid: "8d90d61da62549208fe117b4384a469b"
I20260812 06:37:25.539852 11293 raft_consensus.cc:3060] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [term 26 FOLLOWER]: Advancing to term 27
I20260812 06:37:25.539852  6238 raft_consensus.cc:3060] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b [term 26 FOLLOWER]: Advancing to term 27
I20260812 06:37:25.540736  6700 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 222b20de5a1145c6b8a11df8374e8790 in term 27.
I20260812 06:37:25.540736  7318 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 222b20de5a1145c6b8a11df8374e8790 in term 27.
I20260812 06:37:25.540946  6238 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 222b20de5a1145c6b8a11df8374e8790 in term 27.
I20260812 06:37:25.540937 11293 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 222b20de5a1145c6b8a11df8374e8790 in term 27.
I20260812 06:37:25.541204  9315 leader_election.cc:304] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 7 voters: 4 yes votes; 0 no votes. yes voters: 222b20de5a1145c6b8a11df8374e8790, 42895a9bcba64aa484f6509605035d02, 8d90d61da62549208fe117b4384a469b, b037d17b21f8422fa5e41b70eb4dcd08; no voters: 
I20260812 06:37:25.541442 11635 raft_consensus.cc:2804] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [term 27 FOLLOWER]: Leader election won for term 27
I20260812 06:37:25.541468  6048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "222b20de5a1145c6b8a11df8374e8790" candidate_term: 27 candidate_status { last_received { term: 25 index: 3953 } } ignore_live_leader: false dest_uuid: "ec6c8d818e744cdbb6b0d15b321010e0"
I20260812 06:37:25.541623  6048 raft_consensus.cc:3060] T 7065bb403e3047dd9a0853c49318ef1f P ec6c8d818e744cdbb6b0d15b321010e0 [term 26 FOLLOWER]: Advancing to term 27
I20260812 06:37:25.541661 11635 raft_consensus.cc:697] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [term 27 LEADER]: Becoming Leader. State: Replica: 222b20de5a1145c6b8a11df8374e8790, State: Running, Role: LEADER
I20260812 06:37:25.541872 11635 consensus_queue.cc:237] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3945, Committed index: 3945, Last appended: 25.3953, Last appended by leader: 3953, Current term: 27, Majority size: 4, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df4308c852240859ada53d216ab69ae" member_type: VOTER last_known_addr { host: "127.4.177.6" port: 34269 } } peers { permanent_uuid: "42895a9bcba64aa484f6509605035d02" member_type: VOTER last_known_addr { host: "127.4.177.4" port: 38181 } } peers { permanent_uuid: "222b20de5a1145c6b8a11df8374e8790" member_type: VOTER last_known_addr { host: "127.4.177.5" port: 43521 } } peers { permanent_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 40897 } } peers { permanent_uuid: "8d90d61da62549208fe117b4384a469b" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 41539 } } peers { permanent_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" member_type: VOTER last_known_addr { host: "127.4.177.7" port: 46299 } } peers { permanent_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 41057 } }
I20260812 06:37:25.542850  6048 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P ec6c8d818e744cdbb6b0d15b321010e0 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 222b20de5a1145c6b8a11df8374e8790 in term 27.
I20260812 06:37:25.544420  5814 catalog_manager.cc:5719] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 reported cstate change: term changed from 25 to 27, leader changed from 2df4308c852240859ada53d216ab69ae (127.4.177.6) to 222b20de5a1145c6b8a11df8374e8790 (127.4.177.5). New cstate: current_term: 27 leader_uuid: "222b20de5a1145c6b8a11df8374e8790" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df4308c852240859ada53d216ab69ae" member_type: VOTER last_known_addr { host: "127.4.177.6" port: 34269 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "42895a9bcba64aa484f6509605035d02" member_type: VOTER last_known_addr { host: "127.4.177.4" port: 38181 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "222b20de5a1145c6b8a11df8374e8790" member_type: VOTER last_known_addr { host: "127.4.177.5" port: 43521 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 40897 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8d90d61da62549208fe117b4384a469b" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 41539 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" member_type: VOTER last_known_addr { host: "127.4.177.7" port: 46299 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 41057 } health_report { overall_health: UNKNOWN } } }
W20260812 06:37:25.643316  7211 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.4.177.6:34269 timed out after 1.521s (SENT) [suppressed 1 similar messages]
I20260812 06:37:25.939584  6700 raft_consensus.cc:1275] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [term 27 FOLLOWER]: Refusing update from remote peer 222b20de5a1145c6b8a11df8374e8790: Log matching property violated. Preceding OpId in replica: term: 25 index: 3953. Preceding OpId from leader: term: 27 index: 3954. (index mismatch)
I20260812 06:37:25.940186 11635 consensus_queue.cc:1048] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [LEADER]: Connected to new peer: Peer: permanent_uuid: "42895a9bcba64aa484f6509605035d02" member_type: VOTER last_known_addr { host: "127.4.177.4" port: 38181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3954, Last known committed idx: 3945, Time since last communication: 0.000s
I20260812 06:37:25.960860  7318 raft_consensus.cc:1275] T 7065bb403e3047dd9a0853c49318ef1f P b037d17b21f8422fa5e41b70eb4dcd08 [term 27 FOLLOWER]: Refusing update from remote peer 222b20de5a1145c6b8a11df8374e8790: Log matching property violated. Preceding OpId in replica: term: 25 index: 3918. Preceding OpId from leader: term: 27 index: 3954. (index mismatch)
I20260812 06:37:25.961455 11635 consensus_queue.cc:1048] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" member_type: VOTER last_known_addr { host: "127.4.177.7" port: 46299 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3954, Last known committed idx: 3911, Time since last communication: 0.000s
W20260812 06:37:25.984684  9714 connection.cc:256] RPC call timeout handler was delayed by -3.1461888989433646s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1s for a response
I20260812 06:37:25.984635  9796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "222b20de5a1145c6b8a11df8374e8790" candidate_term: 26 candidate_status { last_received { term: 25 index: 3953 } } ignore_live_leader: false dest_uuid: "2df4308c852240859ada53d216ab69ae" is_pre_election: true
W20260812 06:37:25.984772  9714 connection.cc:256] RPC call timeout handler was delayed by -2.9286853689700365s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1s for a response
I20260812 06:37:25.984716  9799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "222b20de5a1145c6b8a11df8374e8790" candidate_term: 26 candidate_status { last_received { term: 25 index: 3953 } } ignore_live_leader: false dest_uuid: "2df4308c852240859ada53d216ab69ae"
W20260812 06:37:25.984794  9710 connection.cc:256] RPC call timeout handler was delayed by -3.039230763213709s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1s for a response
I20260812 06:37:25.985001  9802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "222b20de5a1145c6b8a11df8374e8790" candidate_term: 26 candidate_status { last_received { term: 25 index: 3953 } } ignore_live_leader: false dest_uuid: "2df4308c852240859ada53d216ab69ae"
I20260812 06:37:25.985213  9804 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "8d90d61da62549208fe117b4384a469b" candidate_term: 26 candidate_status { last_received { term: 25 index: 3948 } } ignore_live_leader: false dest_uuid: "2df4308c852240859ada53d216ab69ae" is_pre_election: true
I20260812 06:37:25.985234  9798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" candidate_term: 26 candidate_status { last_received { term: 25 index: 3918 } } ignore_live_leader: false dest_uuid: "2df4308c852240859ada53d216ab69ae" is_pre_election: true
I20260812 06:37:25.985386  9787 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "42895a9bcba64aa484f6509605035d02" candidate_term: 26 candidate_status { last_received { term: 25 index: 3953 } } ignore_live_leader: false dest_uuid: "2df4308c852240859ada53d216ab69ae"
I20260812 06:37:25.985641  9789 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "42895a9bcba64aa484f6509605035d02" candidate_term: 26 candidate_status { last_received { term: 25 index: 3953 } } ignore_live_leader: false dest_uuid: "2df4308c852240859ada53d216ab69ae"
I20260812 06:37:25.985720  9814 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "42895a9bcba64aa484f6509605035d02" candidate_term: 26 candidate_status { last_received { term: 25 index: 3953 } } ignore_live_leader: false dest_uuid: "2df4308c852240859ada53d216ab69ae" is_pre_election: true
I20260812 06:37:25.984658  9800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "222b20de5a1145c6b8a11df8374e8790" candidate_term: 27 candidate_status { last_received { term: 25 index: 3953 } } ignore_live_leader: false dest_uuid: "2df4308c852240859ada53d216ab69ae"
I20260812 06:37:25.984660  9794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "222b20de5a1145c6b8a11df8374e8790" candidate_term: 26 candidate_status { last_received { term: 25 index: 3953 } } ignore_live_leader: false dest_uuid: "2df4308c852240859ada53d216ab69ae" is_pre_election: true
I20260812 06:37:25.986022  9799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "42895a9bcba64aa484f6509605035d02" candidate_term: 26 candidate_status { last_received { term: 25 index: 3953 } } ignore_live_leader: false dest_uuid: "2df4308c852240859ada53d216ab69ae" is_pre_election: true
I20260812 06:37:25.987717  9425 raft_consensus.cc:1240] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [term 27 LEADER]: Rejecting Update request from peer 2df4308c852240859ada53d216ab69ae for earlier term 25. Current term is 27. Ops: []
I20260812 06:37:25.988220 11519 consensus_queue.cc:1059] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "222b20de5a1145c6b8a11df8374e8790" member_type: VOTER last_known_addr { host: "127.4.177.5" port: 43521 }, Status: INVALID_TERM, Last received: 25.3953, Next index: 3954, Last known committed idx: 3945, Time since last communication: 0.000s
I20260812 06:37:25.988390 11519 consensus_queue.cc:1243] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [LEADER]: Peer 222b20de5a1145c6b8a11df8374e8790 log is divergent from this leader: its last log entry 27.3954 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3945
I20260812 06:37:25.988451 11519 raft_consensus.cc:3055] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [term 25 LEADER]: Stepping down as leader of term 25
I20260812 06:37:25.988483 11519 raft_consensus.cc:740] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [term 25 LEADER]: Becoming Follower/Learner. State: Replica: 2df4308c852240859ada53d216ab69ae, State: Running, Role: LEADER
I20260812 06:37:25.988605 11519 consensus_queue.cc:260] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3945, Committed index: 3945, Last appended: 25.3953, Last appended by leader: 3953, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2df4308c852240859ada53d216ab69ae" member_type: VOTER last_known_addr { host: "127.4.177.6" port: 34269 } } peers { permanent_uuid: "42895a9bcba64aa484f6509605035d02" member_type: VOTER last_known_addr { host: "127.4.177.4" port: 38181 } } peers { permanent_uuid: "222b20de5a1145c6b8a11df8374e8790" member_type: VOTER last_known_addr { host: "127.4.177.5" port: 43521 } } peers { permanent_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 40897 } } peers { permanent_uuid: "8d90d61da62549208fe117b4384a469b" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 41539 } } peers { permanent_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" member_type: VOTER last_known_addr { host: "127.4.177.7" port: 46299 } } peers { permanent_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 41057 } }
W20260812 06:37:25.988979 11587 consensus_queue.cc:1175] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" responder_term: 25 status { last_received { term: 25 index: 3918 } last_committed_idx: 3911 last_received_current_leader { term: 25 index: 3918 } }
I20260812 06:37:25.989194  9790 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "222b20de5a1145c6b8a11df8374e8790" candidate_term: 27 candidate_status { last_received { term: 25 index: 3953 } } ignore_live_leader: false dest_uuid: "2df4308c852240859ada53d216ab69ae" is_pre_election: true
I20260812 06:37:25.989282 11519 raft_consensus.cc:3060] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [term 25 FOLLOWER]: Advancing to term 27
I20260812 06:37:25.989641  9785 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "b037d17b21f8422fa5e41b70eb4dcd08" candidate_term: 26 candidate_status { last_received { term: 25 index: 3918 } } ignore_live_leader: false dest_uuid: "2df4308c852240859ada53d216ab69ae" is_pre_election: true
W20260812 06:37:25.989984 11563 consensus_queue.cc:1175] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "8d90d61da62549208fe117b4384a469b" responder_term: 25 status { last_received { term: 25 index: 3948 } last_committed_idx: 3942 last_received_current_leader { term: 25 index: 3948 } }
I20260812 06:37:25.990121  6700 raft_consensus.cc:1240] T 7065bb403e3047dd9a0853c49318ef1f P 42895a9bcba64aa484f6509605035d02 [term 27 FOLLOWER]: Rejecting Update request from peer 2df4308c852240859ada53d216ab69ae for earlier term 25. Current term is 27. Ops: []
I20260812 06:37:25.990942  9802 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" candidate_term: 26 candidate_status { last_received { term: 25 index: 3891 } } ignore_live_leader: false dest_uuid: "2df4308c852240859ada53d216ab69ae" is_pre_election: true
I20260812 06:37:25.991163  9790 raft_consensus.cc:2468] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 222b20de5a1145c6b8a11df8374e8790 in term 27.
I20260812 06:37:25.991278  6048 raft_consensus.cc:1240] T 7065bb403e3047dd9a0853c49318ef1f P ec6c8d818e744cdbb6b0d15b321010e0 [term 27 FOLLOWER]: Rejecting Update request from peer 2df4308c852240859ada53d216ab69ae for earlier term 25. Current term is 27. Ops: [25.3892-25.3953]
I20260812 06:37:25.992668  9796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7065bb403e3047dd9a0853c49318ef1f" candidate_uuid: "8d90d61da62549208fe117b4384a469b" candidate_term: 26 candidate_status { last_received { term: 25 index: 3948 } } ignore_live_leader: false dest_uuid: "2df4308c852240859ada53d216ab69ae" is_pre_election: true
I20260812 06:37:25.992812  9796 raft_consensus.cc:2368] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8d90d61da62549208fe117b4384a469b for earlier term 26. Current term is 27.
I20260812 06:37:25.996170 11293 raft_consensus.cc:1240] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [term 27 FOLLOWER]: Rejecting Update request from peer 2df4308c852240859ada53d216ab69ae for earlier term 25. Current term is 27. Ops: [25.3727-25.3953]
I20260812 06:37:26.075102  9796 raft_consensus.cc:1275] T 7065bb403e3047dd9a0853c49318ef1f P 2df4308c852240859ada53d216ab69ae [term 27 FOLLOWER]: Refusing update from remote peer 222b20de5a1145c6b8a11df8374e8790: Log matching property violated. Preceding OpId in replica: term: 25 index: 3953. Preceding OpId from leader: term: 27 index: 3954. (index mismatch)
I20260812 06:37:26.075857 11635 consensus_queue.cc:1048] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2df4308c852240859ada53d216ab69ae" member_type: VOTER last_known_addr { host: "127.4.177.6" port: 34269 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3954, Last known committed idx: 3945, Time since last communication: 0.000s
I20260812 06:37:26.080309  6048 raft_consensus.cc:1275] T 7065bb403e3047dd9a0853c49318ef1f P ec6c8d818e744cdbb6b0d15b321010e0 [term 27 FOLLOWER]: Refusing update from remote peer 222b20de5a1145c6b8a11df8374e8790: Log matching property violated. Preceding OpId in replica: term: 25 index: 3891. Preceding OpId from leader: term: 27 index: 3954. (index mismatch)
I20260812 06:37:26.081854 11635 consensus_queue.cc:1048] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec6c8d818e744cdbb6b0d15b321010e0" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 41057 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3954, Last known committed idx: 3884, Time since last communication: 0.000s
I20260812 06:37:26.082041 11686 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:41528 (ReqId={client: 3067f3aef2a44cc4bb7f2ccd6acf83db, seq_no=3925, attempt_no=0}) took 3930 ms. Trace:
I20260812 06:37:26.082123 11686 rpcz_store.cc:276] 0812 06:37:22.151311 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 06:37:22.151355 (+    44us) service_pool.cc:224] Handling call
0812 06:37:26.082022 (+3930667us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0812 06:37:22.152485 (+     0us) write_op.cc:183] PREPARE: starting on tablet 7065bb403e3047dd9a0853c49318ef1f
0812 06:37:22.152564 (+    79us) write_op.cc:432] Acquiring schema lock in shared mode
0812 06:37:22.152567 (+     3us) write_op.cc:435] Acquired schema lock
0812 06:37:22.152569 (+     2us) tablet.cc:662] Decoding operations
0812 06:37:22.152665 (+    96us) write_op.cc:620] Acquiring the partition lock for write op
0812 06:37:22.152671 (+     6us) write_op.cc:641] Partition lock acquired for write op
0812 06:37:22.152674 (+     3us) tablet.cc:685] Acquiring locks for 10 operations
0812 06:37:22.152735 (+    61us) tablet.cc:701] Row locks acquired
0812 06:37:22.152736 (+     1us) write_op.cc:260] PREPARE: finished
0812 06:37:22.152764 (+    28us) write_op.cc:270] Start()
0812 06:37:22.152782 (+    18us) write_op.cc:276] Timestamp: P: 1786516642152725 usec, L: 0
0812 06:37:22.152784 (+     2us) op_driver.cc:348] REPLICATION: starting
0812 06:37:22.152890 (+   106us) log.cc:844] Serialized 686 byte log entry
0812 06:37:26.079000 (+3926110us) op_driver.cc:464] REPLICATION: finished
0812 06:37:26.081408 (+  2408us) write_op.cc:301] APPLY: starting
0812 06:37:26.081421 (+    13us) tablet.cc:1367] starting BulkCheckPresence
0812 06:37:26.081450 (+    29us) tablet.cc:1370] finished BulkCheckPresence
0812 06:37:26.081462 (+    12us) tablet.cc:1372] starting ApplyRowOperation cycle
0812 06:37:26.081698 (+   236us) tablet.cc:1383] finished ApplyRowOperation cycle
0812 06:37:26.081759 (+    61us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0812 06:37:26.081768 (+     9us) write_op.cc:312] APPLY: finished
0812 06:37:26.081843 (+    75us) log.cc:844] Serialized 77 byte log entry
0812 06:37:26.081915 (+    72us) write_op.cc:489] Releasing partition, row and schema locks
0812 06:37:26.081937 (+    22us) write_op.cc:454] Released schema lock
0812 06:37:26.081943 (+     6us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":2344,"num_ops":10,"prepare.queue_time_us":818,"prepare.run_cpu_time_us":487,"prepare.run_wall_time_us":750,"raft.queue_time_us":29,"raft.run_cpu_time_us":4,"raft.run_wall_time_us":6,"replication_time_us":3926185,"thread_start_us":72,"threads_started":1}]]}
I20260812 06:37:26.083145 11686 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:41528 (ReqId={client: 3067f3aef2a44cc4bb7f2ccd6acf83db, seq_no=3927, attempt_no=0}) took 3931 ms. Trace:
I20260812 06:37:26.083225 11686 rpcz_store.cc:276] 0812 06:37:22.151455 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 06:37:22.152317 (+   862us) service_pool.cc:224] Handling call
0812 06:37:26.083131 (+3930814us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0812 06:37:22.153377 (+     0us) write_op.cc:183] PREPARE: starting on tablet 7065bb403e3047dd9a0853c49318ef1f
0812 06:37:22.153459 (+    82us) write_op.cc:432] Acquiring schema lock in shared mode
0812 06:37:22.153462 (+     3us) write_op.cc:435] Acquired schema lock
0812 06:37:22.153464 (+     2us) tablet.cc:662] Decoding operations
0812 06:37:22.153544 (+    80us) write_op.cc:620] Acquiring the partition lock for write op
0812 06:37:22.153548 (+     4us) write_op.cc:641] Partition lock acquired for write op
0812 06:37:22.153550 (+     2us) tablet.cc:685] Acquiring locks for 10 operations
0812 06:37:22.153594 (+    44us) tablet.cc:701] Row locks acquired
0812 06:37:22.153595 (+     1us) write_op.cc:260] PREPARE: finished
0812 06:37:22.153619 (+    24us) write_op.cc:270] Start()
0812 06:37:22.153634 (+    15us) write_op.cc:276] Timestamp: P: 1786516642153581 usec, L: 0
0812 06:37:22.153636 (+     2us) op_driver.cc:348] REPLICATION: starting
0812 06:37:22.153723 (+    87us) log.cc:844] Serialized 696 byte log entry
0812 06:37:26.079158 (+3925435us) op_driver.cc:464] REPLICATION: finished
0812 06:37:26.082644 (+  3486us) write_op.cc:301] APPLY: starting
0812 06:37:26.082657 (+    13us) tablet.cc:1367] starting BulkCheckPresence
0812 06:37:26.082691 (+    34us) tablet.cc:1370] finished BulkCheckPresence
0812 06:37:26.082703 (+    12us) tablet.cc:1372] starting ApplyRowOperation cycle
0812 06:37:26.082881 (+   178us) tablet.cc:1383] finished ApplyRowOperation cycle
0812 06:37:26.082923 (+    42us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0812 06:37:26.082928 (+     5us) write_op.cc:312] APPLY: finished
0812 06:37:26.082994 (+    66us) log.cc:844] Serialized 77 byte log entry
0812 06:37:26.083047 (+    53us) write_op.cc:489] Releasing partition, row and schema locks
0812 06:37:26.083062 (+    15us) write_op.cc:454] Released schema lock
0812 06:37:26.083067 (+     5us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":3447,"num_ops":10,"prepare.queue_time_us":655,"prepare.run_cpu_time_us":467,"prepare.run_wall_time_us":796,"raft.queue_time_us":242,"raft.run_cpu_time_us":5,"raft.run_wall_time_us":5,"replication_time_us":3925501,"thread_start_us":68,"threads_started":1}]]}
I20260812 06:37:26.084123 11686 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:41528 (ReqId={client: 3067f3aef2a44cc4bb7f2ccd6acf83db, seq_no=3926, attempt_no=0}) took 3932 ms. Trace:
I20260812 06:37:26.084185 11686 rpcz_store.cc:276] 0812 06:37:22.151396 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 06:37:22.152846 (+  1450us) service_pool.cc:224] Handling call
0812 06:37:26.084112 (+3931266us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0812 06:37:22.154073 (+     0us) write_op.cc:183] PREPARE: starting on tablet 7065bb403e3047dd9a0853c49318ef1f
0812 06:37:22.154150 (+    77us) write_op.cc:432] Acquiring schema lock in shared mode
0812 06:37:22.154152 (+     2us) write_op.cc:435] Acquired schema lock
0812 06:37:22.154153 (+     1us) tablet.cc:662] Decoding operations
0812 06:37:22.154263 (+   110us) write_op.cc:620] Acquiring the partition lock for write op
0812 06:37:22.154269 (+     6us) write_op.cc:641] Partition lock acquired for write op
0812 06:37:22.154271 (+     2us) tablet.cc:685] Acquiring locks for 10 operations
0812 06:37:22.154332 (+    61us) tablet.cc:701] Row locks acquired
0812 06:37:22.154333 (+     1us) write_op.cc:260] PREPARE: finished
0812 06:37:22.154357 (+    24us) write_op.cc:270] Start()
0812 06:37:22.154372 (+    15us) write_op.cc:276] Timestamp: P: 1786516642154319 usec, L: 0
0812 06:37:22.154373 (+     1us) op_driver.cc:348] REPLICATION: starting
0812 06:37:22.154462 (+    89us) log.cc:844] Serialized 696 byte log entry
0812 06:37:26.079292 (+3924830us) op_driver.cc:464] REPLICATION: finished
0812 06:37:26.083614 (+  4322us) write_op.cc:301] APPLY: starting
0812 06:37:26.083624 (+    10us) tablet.cc:1367] starting BulkCheckPresence
0812 06:37:26.083654 (+    30us) tablet.cc:1370] finished BulkCheckPresence
0812 06:37:26.083666 (+    12us) tablet.cc:1372] starting ApplyRowOperation cycle
0812 06:37:26.083911 (+   245us) tablet.cc:1383] finished ApplyRowOperation cycle
0812 06:37:26.083940 (+    29us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0812 06:37:26.083945 (+     5us) write_op.cc:312] APPLY: finished
0812 06:37:26.084000 (+    55us) log.cc:844] Serialized 77 byte log entry
0812 06:37:26.084035 (+    35us) write_op.cc:489] Releasing partition, row and schema locks
0812 06:37:26.084049 (+    14us) write_op.cc:454] Released schema lock
0812 06:37:26.084057 (+     8us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":4289,"num_ops":10,"prepare.queue_time_us":985,"prepare.run_cpu_time_us":489,"prepare.run_wall_time_us":889,"raft.queue_time_us":37,"raft.run_cpu_time_us":4,"raft.run_wall_time_us":4,"replication_time_us":3924900,"thread_start_us":87,"threads_started":1}]]}
I20260812 06:37:26.084996 11686 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:41528 (ReqId={client: 3067f3aef2a44cc4bb7f2ccd6acf83db, seq_no=3928, attempt_no=0}) took 3933 ms. Trace:
I20260812 06:37:26.085073 11686 rpcz_store.cc:276] 0812 06:37:22.151509 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 06:37:22.153857 (+  2348us) service_pool.cc:224] Handling call
0812 06:37:26.084986 (+3931129us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0812 06:37:22.154995 (+     0us) write_op.cc:183] PREPARE: starting on tablet 7065bb403e3047dd9a0853c49318ef1f
0812 06:37:22.155063 (+    68us) write_op.cc:432] Acquiring schema lock in shared mode
0812 06:37:22.155067 (+     4us) write_op.cc:435] Acquired schema lock
0812 06:37:22.155068 (+     1us) tablet.cc:662] Decoding operations
0812 06:37:22.155166 (+    98us) write_op.cc:620] Acquiring the partition lock for write op
0812 06:37:22.155172 (+     6us) write_op.cc:641] Partition lock acquired for write op
0812 06:37:22.155175 (+     3us) tablet.cc:685] Acquiring locks for 10 operations
0812 06:37:22.155228 (+    53us) tablet.cc:701] Row locks acquired
0812 06:37:22.155229 (+     1us) write_op.cc:260] PREPARE: finished
0812 06:37:22.155253 (+    24us) write_op.cc:270] Start()
0812 06:37:22.155268 (+    15us) write_op.cc:276] Timestamp: P: 1786516642155215 usec, L: 0
0812 06:37:22.155269 (+     1us) op_driver.cc:348] REPLICATION: starting
0812 06:37:22.155351 (+    82us) log.cc:844] Serialized 696 byte log entry
0812 06:37:26.079441 (+3924090us) op_driver.cc:464] REPLICATION: finished
0812 06:37:26.084553 (+  5112us) write_op.cc:301] APPLY: starting
0812 06:37:26.084562 (+     9us) tablet.cc:1367] starting BulkCheckPresence
0812 06:37:26.084595 (+    33us) tablet.cc:1370] finished BulkCheckPresence
0812 06:37:26.084608 (+    13us) tablet.cc:1372] starting ApplyRowOperation cycle
0812 06:37:26.084784 (+   176us) tablet.cc:1383] finished ApplyRowOperation cycle
0812 06:37:26.084812 (+    28us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0812 06:37:26.084817 (+     5us) write_op.cc:312] APPLY: finished
0812 06:37:26.084870 (+    53us) log.cc:844] Serialized 77 byte log entry
0812 06:37:26.084917 (+    47us) write_op.cc:489] Releasing partition, row and schema locks
0812 06:37:26.084929 (+    12us) write_op.cc:454] Released schema lock
0812 06:37:26.084933 (+     4us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":5080,"num_ops":10,"prepare.queue_time_us":905,"prepare.run_cpu_time_us":452,"prepare.run_wall_time_us":452,"raft.queue_time_us":50,"raft.run_cpu_time_us":7,"raft.run_wall_time_us":8,"replication_time_us":3924153,"thread_start_us":93,"threads_started":1}]]}
I20260812 06:37:26.085912 11686 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:41528 (ReqId={client: 3067f3aef2a44cc4bb7f2ccd6acf83db, seq_no=3929, attempt_no=0}) took 3932 ms. Trace:
I20260812 06:37:26.086316 11687 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:41528 (ReqId={client: 3067f3aef2a44cc4bb7f2ccd6acf83db, seq_no=3931, attempt_no=0}) took 3923 ms. Trace:
I20260812 06:37:26.087308 11684 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:41528 (ReqId={client: 3067f3aef2a44cc4bb7f2ccd6acf83db, seq_no=3932, attempt_no=0}) took 3924 ms. Trace:
I20260812 06:37:26.087168 11687 rpcz_store.cc:276] 0812 06:37:22.163071 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 06:37:22.163120 (+    49us) service_pool.cc:224] Handling call
0812 06:37:26.086302 (+3923182us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0812 06:37:22.164128 (+     0us) write_op.cc:183] PREPARE: starting on tablet 7065bb403e3047dd9a0853c49318ef1f
0812 06:37:22.164213 (+    85us) write_op.cc:432] Acquiring schema lock in shared mode
0812 06:37:22.164216 (+     3us) write_op.cc:435] Acquired schema lock
0812 06:37:22.164218 (+     2us) tablet.cc:662] Decoding operations
0812 06:37:22.164310 (+    92us) write_op.cc:620] Acquiring the partition lock for write op
0812 06:37:22.164323 (+    13us) write_op.cc:641] Partition lock acquired for write op
0812 06:37:22.164326 (+     3us) tablet.cc:685] Acquiring locks for 10 operations
0812 06:37:22.164373 (+    47us) tablet.cc:701] Row locks acquired
0812 06:37:22.164373 (+     0us) write_op.cc:260] PREPARE: finished
0812 06:37:22.164398 (+    25us) write_op.cc:270] Start()
0812 06:37:22.164413 (+    15us) write_op.cc:276] Timestamp: P: 1786516642164359 usec, L: 0
0812 06:37:22.164415 (+     2us) op_driver.cc:348] REPLICATION: starting
0812 06:37:22.164512 (+    97us) log.cc:844] Serialized 696 byte log entry
0812 06:37:26.079632 (+3915120us) op_driver.cc:464] REPLICATION: finished
0812 06:37:26.085803 (+  6171us) write_op.cc:301] APPLY: starting
0812 06:37:26.085816 (+    13us) tablet.cc:1367] starting BulkCheckPresence
0812 06:37:26.085856 (+    40us) tablet.cc:1370] finished BulkCheckPresence
0812 06:37:26.085870 (+    14us) tablet.cc:1372] starting ApplyRowOperation cycle
0812 06:37:26.086057 (+   187us) tablet.cc:1383] finished ApplyRowOperation cycle
0812 06:37:26.086086 (+    29us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0812 06:37:26.086095 (+     9us) write_op.cc:312] APPLY: finished
0812 06:37:26.086163 (+    68us) log.cc:844] Serialized 77 byte log entry
0812 06:37:26.086199 (+    36us) write_op.cc:489] Releasing partition, row and schema locks
0812 06:37:26.086236 (+    37us) write_op.cc:454] Released schema lock
0812 06:37:26.086241 (+     5us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":6121,"num_ops":10,"prepare.queue_time_us":679,"prepare.run_cpu_time_us":496,"prepare.run_wall_time_us":543,"raft.queue_time_us":34,"raft.run_cpu_time_us":5,"raft.run_wall_time_us":5,"replication_time_us":3915207}]]}
I20260812 06:37:26.087021 11681 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:41528 (ReqId={client: 3067f3aef2a44cc4bb7f2ccd6acf83db, seq_no=3930, attempt_no=0}) took 3924 ms. Trace:
I20260812 06:37:26.087976 11681 rpcz_store.cc:276] 0812 06:37:22.162931 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 06:37:22.164960 (+  2029us) service_pool.cc:224] Handling call
0812 06:37:26.086995 (+3922035us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0812 06:37:22.165268 (+     0us) write_op.cc:183] PREPARE: starting on tablet 7065bb403e3047dd9a0853c49318ef1f
0812 06:37:22.165337 (+    69us) write_op.cc:432] Acquiring schema lock in shared mode
0812 06:37:22.165342 (+     5us) write_op.cc:435] Acquired schema lock
0812 06:37:22.165344 (+     2us) tablet.cc:662] Decoding operations
0812 06:37:22.165429 (+    85us) write_op.cc:620] Acquiring the partition lock for write op
0812 06:37:22.165433 (+     4us) write_op.cc:641] Partition lock acquired for write op
0812 06:37:22.165435 (+     2us) tablet.cc:685] Acquiring locks for 10 operations
0812 06:37:22.165481 (+    46us) tablet.cc:701] Row locks acquired
0812 06:37:22.165482 (+     1us) write_op.cc:260] PREPARE: finished
0812 06:37:22.165504 (+    22us) write_op.cc:270] Start()
0812 06:37:22.165519 (+    15us) write_op.cc:276] Timestamp: P: 1786516642165466 usec, L: 0
0812 06:37:22.165520 (+     1us) op_driver.cc:348] REPLICATION: starting
0812 06:37:22.165601 (+    81us) log.cc:844] Serialized 696 byte log entry
0812 06:37:26.079688 (+3914087us) op_driver.cc:464] REPLICATION: finished
0812 06:37:26.086561 (+  6873us) write_op.cc:301] APPLY: starting
0812 06:37:26.086572 (+    11us) tablet.cc:1367] starting BulkCheckPresence
0812 06:37:26.086607 (+    35us) tablet.cc:1370] finished BulkCheckPresence
0812 06:37:26.086617 (+    10us) tablet.cc:1372] starting ApplyRowOperation cycle
0812 06:37:26.086785 (+   168us) tablet.cc:1383] finished ApplyRowOperation cycle
0812 06:37:26.086811 (+    26us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0812 06:37:26.086814 (+     3us) write_op.cc:312] APPLY: finished
0812 06:37:26.086875 (+    61us) log.cc:844] Serialized 77 byte log entry
0812 06:37:26.086920 (+    45us) write_op.cc:489] Releasing partition, row and schema locks
0812 06:37:26.086932 (+    12us) write_op.cc:454] Released schema lock
0812 06:37:26.086938 (+     6us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":6836,"num_ops":10,"prepare.queue_time_us":80,"prepare.run_cpu_time_us":429,"prepare.run_wall_time_us":554,"raft.queue_time_us":107,"raft.run_cpu_time_us":4,"raft.run_wall_time_us":5,"replication_time_us":3914157}]]}
I20260812 06:37:26.088967  6238 raft_consensus.cc:1275] T 7065bb403e3047dd9a0853c49318ef1f P 8d90d61da62549208fe117b4384a469b [term 27 FOLLOWER]: Refusing update from remote peer 222b20de5a1145c6b8a11df8374e8790: Log matching property violated. Preceding OpId in replica: term: 25 index: 3948. Preceding OpId from leader: term: 27 index: 3954. (index mismatch)
W20260812 06:37:26.093659  9738 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:41528: Illegal state: replica 2df4308c852240859ada53d216ab69ae is not leader of this config: current role FOLLOWER
I20260812 06:37:26.094561 11293 raft_consensus.cc:1275] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e [term 27 FOLLOWER]: Refusing update from remote peer 222b20de5a1145c6b8a11df8374e8790: Log matching property violated. Preceding OpId in replica: term: 23 index: 3627. Preceding OpId from leader: term: 27 index: 3954. (index mismatch)
I20260812 06:37:26.087566 11684 rpcz_store.cc:276] 0812 06:37:22.163143 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 06:37:22.163483 (+   340us) service_pool.cc:224] Handling call
0812 06:37:26.087273 (+3923790us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0812 06:37:22.164685 (+     0us) write_op.cc:183] PREPARE: starting on tablet 7065bb403e3047dd9a0853c49318ef1f
0812 06:37:22.164763 (+    78us) write_op.cc:432] Acquiring schema lock in shared mode
0812 06:37:22.164766 (+     3us) write_op.cc:435] Acquired schema lock
0812 06:37:22.164768 (+     2us) tablet.cc:662] Decoding operations
0812 06:37:22.164856 (+    88us) write_op.cc:620] Acquiring the partition lock for write op
0812 06:37:22.164861 (+     5us) write_op.cc:641] Partition lock acquired for write op
0812 06:37:22.164863 (+     2us) tablet.cc:685] Acquiring locks for 10 operations
0812 06:37:22.164934 (+    71us) tablet.cc:701] Row locks acquired
0812 06:37:22.164935 (+     1us) write_op.cc:260] PREPARE: finished
0812 06:37:22.164970 (+    35us) write_op.cc:270] Start()
0812 06:37:22.164989 (+    19us) write_op.cc:276] Timestamp: P: 1786516642164929 usec, L: 0
0812 06:37:22.164991 (+     2us) op_driver.cc:348] REPLICATION: starting
0812 06:37:22.165100 (+   109us) log.cc:844] Serialized 696 byte log entry
0812 06:37:26.079660 (+3914560us) op_driver.cc:464] REPLICATION: finished
0812 06:37:26.086091 (+  6431us) write_op.cc:301] APPLY: starting
0812 06:37:26.086103 (+    12us) tablet.cc:1367] starting BulkCheckPresence
0812 06:37:26.086133 (+    30us) tablet.cc:1370] finished BulkCheckPresence
0812 06:37:26.086146 (+    13us) tablet.cc:1372] starting ApplyRowOperation cycle
0812 06:37:26.086886 (+   740us) tablet.cc:1383] finished ApplyRowOperation cycle
0812 06:37:26.086934 (+    48us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0812 06:37:26.086939 (+     5us) write_op.cc:312] APPLY: finished
0812 06:37:26.087051 (+   112us) log.cc:844] Serialized 77 byte log entry
0812 06:37:26.087127 (+    76us) write_op.cc:489] Releasing partition, row and schema locks
0812 06:37:26.087142 (+    15us) write_op.cc:454] Released schema lock
0812 06:37:26.087179 (+    37us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":6393,"num_ops":10,"prepare.queue_time_us":964,"prepare.run_cpu_time_us":513,"prepare.run_wall_time_us":558,"raft.queue_time_us":31,"raft.run_cpu_time_us":4,"raft.run_wall_time_us":5,"replication_time_us":3914659}]]}
I20260812 06:37:26.087539 11686 rpcz_store.cc:276] 0812 06:37:22.153924 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 06:37:22.154437 (+   513us) service_pool.cc:224] Handling call
0812 06:37:26.085901 (+3931464us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0812 06:37:22.155468 (+     0us) write_op.cc:183] PREPARE: starting on tablet 7065bb403e3047dd9a0853c49318ef1f
0812 06:37:22.155540 (+    72us) write_op.cc:432] Acquiring schema lock in shared mode
0812 06:37:22.155543 (+     3us) write_op.cc:435] Acquired schema lock
0812 06:37:22.155544 (+     1us) tablet.cc:662] Decoding operations
0812 06:37:22.155636 (+    92us) write_op.cc:620] Acquiring the partition lock for write op
0812 06:37:22.155641 (+     5us) write_op.cc:641] Partition lock acquired for write op
0812 06:37:22.155644 (+     3us) tablet.cc:685] Acquiring locks for 10 operations
0812 06:37:22.155700 (+    56us) tablet.cc:701] Row locks acquired
0812 06:37:22.155701 (+     1us) write_op.cc:260] PREPARE: finished
0812 06:37:22.155723 (+    22us) write_op.cc:270] Start()
0812 06:37:22.155738 (+    15us) write_op.cc:276] Timestamp: P: 1786516642155686 usec, L: 0
0812 06:37:22.155739 (+     1us) op_driver.cc:348] REPLICATION: starting
0812 06:37:22.155819 (+    80us) log.cc:844] Serialized 686 byte log entry
0812 06:37:26.079598 (+3923779us) op_driver.cc:464] REPLICATION: finished
0812 06:37:26.085444 (+  5846us) write_op.cc:301] APPLY: starting
0812 06:37:26.085453 (+     9us) tablet.cc:1367] starting BulkCheckPresence
0812 06:37:26.085479 (+    26us) tablet.cc:1370] finished BulkCheckPresence
0812 06:37:26.085491 (+    12us) tablet.cc:1372] starting ApplyRowOperation cycle
0812 06:37:26.085687 (+   196us) tablet.cc:1383] finished ApplyRowOperation cycle
0812 06:37:26.085716 (+    29us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0812 06:37:26.085720 (+     4us) write_op.cc:312] APPLY: finished
0812 06:37:26.085770 (+    50us) log.cc:844] Serialized 77 byte log entry
0812 06:37:26.085809 (+    39us) write_op.cc:489] Releasing partition, row and schema locks
0812 06:37:26.085830 (+    21us) write_op.cc:454] Released schema lock
0812 06:37:26.085835 (+     5us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":5816,"num_ops":10,"prepare.queue_time_us":807,"prepare.run_cpu_time_us":442,"prepare.run_wall_time_us":442,"raft.queue_time_us":60,"raft.run_cpu_time_us":7,"raft.run_wall_time_us":8,"replication_time_us":3923843}]]}
W20260812 06:37:26.095144  9742 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:41528: Illegal state: replica 2df4308c852240859ada53d216ab69ae is not leader of this config: current role FOLLOWER
W20260812 06:37:26.096688  9742 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:41528: Illegal state: replica 2df4308c852240859ada53d216ab69ae is not leader of this config: current role FOLLOWER
W20260812 06:37:26.097133  9749 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:41528: Illegal state: replica 2df4308c852240859ada53d216ab69ae is not leader of this config: current role FOLLOWER
W20260812 06:37:26.097611  9748 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:41528: Illegal state: replica 2df4308c852240859ada53d216ab69ae is not leader of this config: current role FOLLOWER
I20260812 06:37:26.097767 11689 consensus_queue.cc:1048] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8d90d61da62549208fe117b4384a469b" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 41539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3954, Last known committed idx: 3942, Time since last communication: 0.000s
W20260812 06:37:26.097133  9744 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:41528: Illegal state: replica 2df4308c852240859ada53d216ab69ae is not leader of this config: current role FOLLOWER
W20260812 06:37:26.099071  6623 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:42408: Illegal state: replica 42895a9bcba64aa484f6509605035d02 is not leader of this config: current role FOLLOWER
W20260812 06:37:26.099166  9738 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:41528: Illegal state: replica 2df4308c852240859ada53d216ab69ae is not leader of this config: current role FOLLOWER
I20260812 06:37:26.099547 11672 consensus_queue.cc:1048] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7962f587ccd842ad8a3f6ed9a5d7707e" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 40897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3954, Last known committed idx: 3619, Time since last communication: 0.000s
I20260812 06:37:26.099663 11672 consensus_queue.cc:1243] T 7065bb403e3047dd9a0853c49318ef1f P 222b20de5a1145c6b8a11df8374e8790 [LEADER]: Peer 7962f587ccd842ad8a3f6ed9a5d7707e log is divergent from this leader: its last log entry 23.3627 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3619
W20260812 06:37:26.102651  9742 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:41528: Illegal state: replica 2df4308c852240859ada53d216ab69ae is not leader of this config: current role FOLLOWER
W20260812 06:37:26.105934  6623 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:42408: Illegal state: replica 42895a9bcba64aa484f6509605035d02 is not leader of this config: current role FOLLOWER
W20260812 06:37:26.106163  6631 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:42408: Illegal state: replica 42895a9bcba64aa484f6509605035d02 is not leader of this config: current role FOLLOWER
W20260812 06:37:26.106766  6628 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:42408: Illegal state: replica 42895a9bcba64aa484f6509605035d02 is not leader of this config: current role FOLLOWER
W20260812 06:37:26.107242  6625 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:42408: Illegal state: replica 42895a9bcba64aa484f6509605035d02 is not leader of this config: current role FOLLOWER
W20260812 06:37:26.107800  6628 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:42408: Illegal state: replica 42895a9bcba64aa484f6509605035d02 is not leader of this config: current role FOLLOWER
I20260812 06:37:26.110800 11293 pending_rounds.cc:85] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e: Aborting all ops after (but not including) 3619
I20260812 06:37:26.111012 11293 pending_rounds.cc:107] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e: Aborting uncommitted WRITE_OP operation due to leader change: 23.3620
I20260812 06:37:26.111702 11293 pending_rounds.cc:107] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e: Aborting uncommitted WRITE_OP operation due to leader change: 23.3621
I20260812 06:37:26.112002 11293 pending_rounds.cc:107] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e: Aborting uncommitted WRITE_OP operation due to leader change: 23.3622
I20260812 06:37:26.112257 11293 pending_rounds.cc:107] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e: Aborting uncommitted WRITE_OP operation due to leader change: 23.3623
I20260812 06:37:26.128063 11293 pending_rounds.cc:107] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e: Aborting uncommitted WRITE_OP operation due to leader change: 23.3624
I20260812 06:37:26.137140 11293 pending_rounds.cc:107] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e: Aborting uncommitted WRITE_OP operation due to leader change: 23.3625
I20260812 06:37:26.137482 11293 pending_rounds.cc:107] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e: Aborting uncommitted WRITE_OP operation due to leader change: 23.3626
I20260812 06:37:26.137691 11293 pending_rounds.cc:107] T 7065bb403e3047dd9a0853c49318ef1f P 7962f587ccd842ad8a3f6ed9a5d7707e: Aborting uncommitted WRITE_OP operation due to leader change: 23.3627
I20260812 06:37:26.429333 11748 mvcc.cc:204] Tried to move back new op lower bound from 7317572166309748736 to 7317572150747422720. Current Snapshot: MvccSnapshot[applied={T|T < 7317572165491830784}]
I20260812 06:37:26.552838  9393 tablet_service.cc:1467] Tablet server has 1 leaders and 0 scanners
I20260812 06:37:26.552943  9780 tablet_service.cc:1467] Tablet server has 0 leaders and 0 scanners
I20260812 06:37:26.556152  7291 tablet_service.cc:1467] Tablet server has 0 leaders and 0 scanners
I20260812 06:37:26.557014 11260 tablet_service.cc:1467] Tablet server has 0 leaders and 0 scanners
I20260812 06:37:26.558192  6213 tablet_service.cc:1467] Tablet server has 0 leaders and 0 scanners
I20260812 06:37:26.571235  6020 tablet_service.cc:1467] Tablet server has 0 leaders and 0 scanners
I20260812 06:37:26.576987  6667 tablet_service.cc:1467] Tablet server has 0 leaders and 0 scanners
Master Summary
               UUID               |      Address       | Status
----------------------------------+--------------------+---------
 f6078ff727624d5c90a44fd1f23a510c | 127.4.177.62:42183 | HEALTHY

Unusual flags for Master:
               Flag               |                                                                                          Value                                                                                           |      Tags       |         Master
----------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------+-------------------------
 ipki_ca_key_size                 | 768                                                                                                                                                                                      | experimental    | all 1 server(s) checked
 ipki_server_key_size             | 768                                                                                                                                                                                      | experimental    | all 1 server(s) checked
 never_fsync                      | true                                                                                                                                                                                     | unsafe,advanced | all 1 server(s) checked
 openssl_security_level_override  | 0                                                                                                                                                                                        | unsafe,hidden   | all 1 server(s) checked
 rpc_reuseport                    | true                                                                                                                                                                                     | experimental    | all 1 server(s) checked
 rpc_server_allow_ephemeral_ports | true                                                                                                                                                                                     | unsafe          | all 1 server(s) checked
 server_dump_info_format          | pb                                                                                                                                                                                       | hidden          | all 1 server(s) checked
 server_dump_info_path            | /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786516568546459-4804-0/raft_consensus-itest-cluster/master-0/data/info.pb | hidden          | all 1 server(s) checked
 tsk_num_rsa_bits                 | 512                                                                                                                                                                                      | experimental    | all 1 server(s) checked

Flags of checked categories for Master:
        Flag         |       Value        |         Master
---------------------+--------------------+-------------------------
 builtin_ntp_servers | 127.4.177.20:34335 | all 1 server(s) checked
 time_source         | builtin            | all 1 server(s) checked

Tablet Server Summary
               UUID               |      Address      | Status  | Location | Tablet Leaders | Active Scanners
----------------------------------+-------------------+---------+----------+----------------+-----------------
 222b20de5a1145c6b8a11df8374e8790 | 127.4.177.5:43521 | HEALTHY | <none>   |       1        |       0
 2df4308c852240859ada53d216ab69ae | 127.4.177.6:34269 | HEALTHY | <none>   |       0        |       0
 42895a9bcba64aa484f6509605035d02 | 127.4.177.4:38181 | HEALTHY | <none>   |       0        |       0
 7962f587ccd842ad8a3f6ed9a5d7707e | 127.4.177.3:40897 | HEALTHY | <none>   |       0        |       0
 8d90d61da62549208fe117b4384a469b | 127.4.177.2:41539 | HEALTHY | <none>   |       0        |       0
 b037d17b21f8422fa5e41b70eb4dcd08 | 127.4.177.7:46299 | HEALTHY | <none>   |       0        |       0
 ec6c8d818e744cdbb6b0d15b321010e0 | 127.4.177.1:41057 | HEALTHY | <none>   |       0        |       0

Tablet Server Location Summary
 Location |  Count
----------+---------
 <none>   |       7

Unusual flags for Tablet Server:
               Flag               |                                                                                        Value                                                                                         |      Tags       |      Tablet Server
----------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------+-------------------------
 ipki_server_key_size             | 768                                                                                                                                                                                  | experimental    | all 7 server(s) checked
 local_ip_for_outbound_sockets    | 127.4.177.1                                                                                                                                                                          | experimental    | 127.4.177.1:41057
 local_ip_for_outbound_sockets    | 127.4.177.2                                                                                                                                                                          | experimental    | 127.4.177.2:41539
 local_ip_for_outbound_sockets    | 127.4.177.3                                                                                                                                                                          | experimental    | 127.4.177.3:40897
 local_ip_for_outbound_sockets    | 127.4.177.4                                                                                                                                                                          | experimental    | 127.4.177.4:38181
 local_ip_for_outbound_sockets    | 127.4.177.5                                                                                                                                                                          | experimental    | 127.4.177.5:43521
 local_ip_for_outbound_sockets    | 127.4.177.6                                                                                                                                                                          | experimental    | 127.4.177.6:34269
 local_ip_for_outbound_sockets    | 127.4.177.7                                                                                                                                                                          | experimental    | 127.4.177.7:46299
 never_fsync                      | true                                                                                                                                                                                 | unsafe,advanced | all 7 server(s) checked
 openssl_security_level_override  | 0                                                                                                                                                                                    | unsafe,hidden   | all 7 server(s) checked
 rpc_server_allow_ephemeral_ports | true                                                                                                                                                                                 | unsafe          | all 7 server(s) checked
 server_dump_info_format          | pb                                                                                                                                                                                   | hidden          | all 7 server(s) checked
 server_dump_info_path            | /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-0/data/info.pb | hidden          | 127.4.177.1:41057
 server_dump_info_path            | /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-1/data/info.pb | hidden          | 127.4.177.2:41539
 server_dump_info_path            | /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-2/data/info.pb | hidden          | 127.4.177.3:40897
 server_dump_info_path            | /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-3/data/info.pb | hidden          | 127.4.177.4:38181
 server_dump_info_path            | /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-4/data/info.pb | hidden          | 127.4.177.5:43521
 server_dump_info_path            | /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-5/data/info.pb | hidden          | 127.4.177.6:34269
 server_dump_info_path            | /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.MultiThreadedInsertWithFailovers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-6/data/info.pb | hidden          | 127.4.177.7:46299

Flags of checked categories for Tablet Server:
        Flag         |       Value        |      Tablet Server
---------------------+--------------------+-------------------------
 builtin_ntp_servers | 127.4.177.20:34335 | all 7 server(s) checked
 time_source         | builtin            | all 7 server(s) checked

Version Summary
     Version     |         Servers
-----------------+-------------------------
 1.19.0-SNAPSHOT | all 8 server(s) checked

Tablet Summary
The cluster doesn't have any matching system tables
Summary by table
   Name    | RF | Status  | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
-----------+----+---------+---------------+---------+------------+------------------+-------------
 TestTable | 7  | HEALTHY | 1             | 1       | 0          | 0                | 0

Tablet Replica Count Summary
   Statistic    | Replica Count
----------------+---------------
 Minimum        | 1
 First Quartile | 1
 Median         | 1
 Third Quartile | 1
 Maximum        | 1

Total Count Summary
                | Total Count
----------------+-------------
 Masters        | 1
 Tablet Servers | 7
 Tables         | 1
 Tablets        | 1
 Replicas       | 7

==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set

OK
I20260812 06:37:26.605865  4804 log_verifier.cc:126] Checking tablet 7065bb403e3047dd9a0853c49318ef1f
I20260812 06:37:28.542119  4804 log_verifier.cc:177] Verified matching terms for 4021 ops in tablet 7065bb403e3047dd9a0853c49318ef1f
I20260812 06:37:28.671114  4804 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskCnZu03/build/debug/bin/kudu with pid 5910
I20260812 06:37:28.695878  4804 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskCnZu03/build/debug/bin/kudu with pid 6122
I20260812 06:37:28.714882  4804 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskCnZu03/build/debug/bin/kudu with pid 11142
I20260812 06:37:28.729964  4804 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskCnZu03/build/debug/bin/kudu with pid 6561
I20260812 06:37:28.742277  4804 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskCnZu03/build/debug/bin/kudu with pid 9284
I20260812 06:37:28.756336  4804 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskCnZu03/build/debug/bin/kudu with pid 9679
I20260812 06:37:28.771544  4804 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskCnZu03/build/debug/bin/kudu with pid 7182
I20260812 06:37:28.786060  4804 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskCnZu03/build/debug/bin/kudu with pid 5761
2026-08-12T06:37:28Z chronyd exiting
[       OK ] RaftConsensusITest.MultiThreadedInsertWithFailovers (78112 ms)
[ RUN      ] RaftConsensusITest.TestReplaceChangeConfigOperation
I20260812 06:37:28.827229  4804 ts_itest-base.cc:115] Starting cluster with:
I20260812 06:37:28.827296  4804 ts_itest-base.cc:116] --------------
I20260812 06:37:28.827311  4804 ts_itest-base.cc:117] 3 tablet servers
I20260812 06:37:28.827329  4804 ts_itest-base.cc:118] 3 replicas per TS
I20260812 06:37:28.827358  4804 ts_itest-base.cc:119] --------------
2026-08-12T06:37:28Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-08-12T06:37:28Z Disabled control of system clock
I20260812 06:37:28.837792  4804 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskCnZu03/build/debug/bin/kudu
/tmp/dist-test-taskCnZu03/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestReplaceChangeConfigOperation.1786516568546459-4804-0/raft_consensus-itest-cluster/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestReplaceChangeConfigOperation.1786516568546459-4804-0/raft_consensus-itest-cluster/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestReplaceChangeConfigOperation.1786516568546459-4804-0/raft_consensus-itest-cluster/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestReplaceChangeConfigOperation.1786516568546459-4804-0/raft_consensus-itest-cluster/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.177.62:32865
--webserver_interface=127.4.177.62
--webserver_port=0
--builtin_ntp_servers=127.4.177.20:41343
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.177.62:32865
--master_add_server_when_underreplicated=false
--catalog_manager_wait_for_new_tablets_to_elect_leader=false with env {}
W20260812 06:37:28.942898 11857 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:37:28.943147 11857 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:37:28.943223 11857 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 06:37:28.946478 11857 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 06:37:28.946565 11857 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:37:28.946605 11857 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 06:37:28.946664 11857 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260812 06:37:28.950428 11857 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestReplaceChangeConfigOperation.1786516568546459-4804-0/raft_consensus-itest-cluster/master-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.177.20:41343
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestReplaceChangeConfigOperation.1786516568546459-4804-0/raft_consensus-itest-cluster/master-0/data
--fs_wal_dir=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestReplaceChangeConfigOperation.1786516568546459-4804-0/raft_consensus-itest-cluster/master-0/wal
--catalog_manager_wait_for_new_tablets_to_elect_leader=false
--master_add_server_when_underreplicated=false
--ipki_ca_key_size=768
--master_addresses=127.4.177.62:32865
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.177.62:32865
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestReplaceChangeConfigOperation.1786516568546459-4804-0/raft_consensus-itest-cluster/master-0/data/info.pb
--webserver_interface=127.4.177.62
--webserver_port=0
--never_fsync=true
--heap_profile_path=/tmp/kudu.11857
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false

Master server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 06:37:28.951494 11857 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:37:28.952590 11857 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 06:37:28.958140 11871 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:37:28.958397 11857 server_base.cc:1061] running on GCE node
W20260812 06:37:28.958683 11874 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:37:28.958720 11867 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:37:28.959223 11857 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:37:28.960117 11857 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:37:28.961376 11857 hybrid_clock.cc:648] HybridClock initialized: now 1786516648961335 us; error 47 us; skew 500 ppm
I20260812 06:37:28.963302 11857 webserver.cc:533] Webserver started at http://127.4.177.62:36563/ using document root <none> and password file <none>
I20260812 06:37:28.963882 11857 fs_manager.cc:362] Metadata directory not provided
I20260812 06:37:28.963965 11857 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:37:28.964200 11857 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:37:28.965812 11857 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestReplaceChangeConfigOperation.1786516568546459-4804-0/raft_consensus-itest-cluster/master-0/data/instance:
uuid: "159343eef8a04d51b49cdc4a4bedeef0"
format_stamp: "Formatted at 2026-08-12 06:37:28 on dist-test-slave-jztv"
I20260812 06:37:28.966367 11857 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestReplaceChangeConfigOperation.1786516568546459-4804-0/raft_consensus-itest-cluster/master-0/wal/instance:
uuid: "159343eef8a04d51b49cdc4a4bedeef0"
format_stamp: "Formatted at 2026-08-12 06:37:28 on dist-test-slave-jztv"
I20260812 06:37:28.969537 11857 fs_manager.cc:696] Time spent creating directory manager: real 0.003s	user 0.002s	sys 0.000s
I20260812 06:37:28.971649 11882 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 06:37:28.972549 11857 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.000s	sys 0.001s
I20260812 06:37:28.972699 11857 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestReplaceChangeConfigOperation.1786516568546459-4804-0/raft_consensus-itest-cluster/master-0/data,/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestReplaceChangeConfigOperation.1786516568546459-4804-0/raft_consensus-itest-cluster/master-0/wal
uuid: "159343eef8a04d51b49cdc4a4bedeef0"
format_stamp: "Formatted at 2026-08-12 06:37:28 on dist-test-slave-jztv"
I20260812 06:37:28.972828 11857 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestReplaceChangeConfigOperation.1786516568546459-4804-0/raft_consensus-itest-cluster/master-0/wal
metadata directory: /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestReplaceChangeConfigOperation.1786516568546459-4804-0/raft_consensus-itest-cluster/master-0/wal
1 data directories: /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestReplaceChangeConfigOperation.1786516568546459-4804-0/raft_consensus-itest-cluster/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:37:28.999795 11857 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 06:37:29.000478 11857 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:37:29.000689 11857 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:37:29.008123 11969 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.4.177.62:32865 every 8 connection(s)
I20260812 06:37:29.008127 11857 rpc_server.cc:307] RPC server started. Bound to: 127.4.177.62:32865
I20260812 06:37:29.009114 11857 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestReplaceChangeConfigOperation.1786516568546459-4804-0/raft_consensus-itest-cluster/master-0/data/info.pb
I20260812 06:37:29.012374 11971 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:37:29.013460  4804 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskCnZu03/build/debug/bin/kudu as pid 11857
I20260812 06:37:29.013612  4804 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestReplaceChangeConfigOperation.1786516568546459-4804-0/raft_consensus-itest-cluster/master-0/wal/instance
I20260812 06:37:29.017763 11971 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 159343eef8a04d51b49cdc4a4bedeef0: Bootstrap starting.
I20260812 06:37:29.020000 11971 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 159343eef8a04d51b49cdc4a4bedeef0: Neither blocks nor log segments found. Creating new log.
I20260812 06:37:29.020677 11971 log.cc:826] T 00000000000000000000000000000000 P 159343eef8a04d51b49cdc4a4bedeef0: Log is configured to *not* fsync() on all Append() calls
I20260812 06:37:29.022476 11971 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 159343eef8a04d51b49cdc4a4bedeef0: No bootstrap required, opened a new log
I20260812 06:37:29.024833 11971 raft_consensus.cc:359] T 00000000000000000000000000000000 P 159343eef8a04d51b49cdc4a4bedeef0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "159343eef8a04d51b49cdc4a4bedeef0" member_type: VOTER last_known_addr { host: "127.4.177.62" port: 32865 } }
I20260812 06:37:29.025040 11971 raft_consensus.cc:385] T 00000000000000000000000000000000 P 159343eef8a04d51b49cdc4a4bedeef0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:37:29.025130 11971 raft_consensus.cc:740] T 00000000000000000000000000000000 P 159343eef8a04d51b49cdc4a4bedeef0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 159343eef8a04d51b49cdc4a4bedeef0, State: Initialized, Role: FOLLOWER
I20260812 06:37:29.025568 11971 consensus_queue.cc:260] T 00000000000000000000000000000000 P 159343eef8a04d51b49cdc4a4bedeef0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "159343eef8a04d51b49cdc4a4bedeef0" member_type: VOTER last_known_addr { host: "127.4.177.62" port: 32865 } }
I20260812 06:37:29.025710 11971 raft_consensus.cc:399] T 00000000000000000000000000000000 P 159343eef8a04d51b49cdc4a4bedeef0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 06:37:29.025795 11971 raft_consensus.cc:493] T 00000000000000000000000000000000 P 159343eef8a04d51b49cdc4a4bedeef0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 06:37:29.025902 11971 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 159343eef8a04d51b49cdc4a4bedeef0 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:37:29.026685 11971 raft_consensus.cc:515] T 00000000000000000000000000000000 P 159343eef8a04d51b49cdc4a4bedeef0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "159343eef8a04d51b49cdc4a4bedeef0" member_type: VOTER last_known_addr { host: "127.4.177.62" port: 32865 } }
I20260812 06:37:29.027015 11971 leader_election.cc:304] T 00000000000000000000000000000000 P 159343eef8a04d51b49cdc4a4bedeef0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 159343eef8a04d51b49cdc4a4bedeef0; no voters: 
I20260812 06:37:29.027287 11971 leader_election.cc:290] T 00000000000000000000000000000000 P 159343eef8a04d51b49cdc4a4bedeef0 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260812 06:37:29.027489 11981 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 159343eef8a04d51b49cdc4a4bedeef0 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 06:37:29.027738 11981 raft_consensus.cc:697] T 00000000000000000000000000000000 P 159343eef8a04d51b49cdc4a4bedeef0 [term 1 LEADER]: Becoming Leader. State: Replica: 159343eef8a04d51b49cdc4a4bedeef0, State: Running, Role: LEADER
I20260812 06:37:29.028056 11981 consensus_queue.cc:237] T 00000000000000000000000000000000 P 159343eef8a04d51b49cdc4a4bedeef0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "159343eef8a04d51b49cdc4a4bedeef0" member_type: VOTER last_known_addr { host: "127.4.177.62" port: 32865 } }
I20260812 06:37:29.028462 11971 sys_catalog.cc:565] T 00000000000000000000000000000000 P 159343eef8a04d51b49cdc4a4bedeef0 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 06:37:29.029691 11982 sys_catalog.cc:455] T 00000000000000000000000000000000 P 159343eef8a04d51b49cdc4a4bedeef0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "159343eef8a04d51b49cdc4a4bedeef0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "159343eef8a04d51b49cdc4a4bedeef0" member_type: VOTER last_known_addr { host: "127.4.177.62" port: 32865 } } }
I20260812 06:37:29.029696 11983 sys_catalog.cc:455] T 00000000000000000000000000000000 P 159343eef8a04d51b49cdc4a4bedeef0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 159343eef8a04d51b49cdc4a4bedeef0. Latest consensus state: current_term: 1 leader_uuid: "159343eef8a04d51b49cdc4a4bedeef0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "159343eef8a04d51b49cdc4a4bedeef0" member_type: VOTER last_known_addr { host: "127.4.177.62" port: 32865 } } }
I20260812 06:37:29.029934 11982 sys_catalog.cc:458] T 00000000000000000000000000000000 P 159343eef8a04d51b49cdc4a4bedeef0 [sys.catalog]: This master's current role is: LEADER
I20260812 06:37:29.029934 11983 sys_catalog.cc:458] T 00000000000000000000000000000000 P 159343eef8a04d51b49cdc4a4bedeef0 [sys.catalog]: This master's current role is: LEADER
I20260812 06:37:29.030730 11996 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260812 06:37:29.033059 11996 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260812 06:37:29.037927 11996 catalog_manager.cc:1383] Generated new cluster ID: 4c49ae54d58541b9911edf35644bcd03
I20260812 06:37:29.037994 11996 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260812 06:37:29.050868 11996 catalog_manager.cc:1406] Generated new certificate authority record
I20260812 06:37:29.052019 11996 catalog_manager.cc:1540] Loading token signing keys...
I20260812 06:37:29.061650 11996 catalog_manager.cc:6092] T 00000000000000000000000000000000 P 159343eef8a04d51b49cdc4a4bedeef0: Generated new TSK 0
I20260812 06:37:29.062376 11996 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260812 06:37:29.066449  4804 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskCnZu03/build/debug/bin/kudu
/tmp/dist-test-taskCnZu03/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestReplaceChangeConfigOperation.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestReplaceChangeConfigOperation.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestReplaceChangeConfigOperation.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestReplaceChangeConfigOperation.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.177.1:0
--local_ip_for_outbound_sockets=127.4.177.1
--webserver_interface=127.4.177.1
--webserver_port=0
--tserver_master_addrs=127.4.177.62:32865
--builtin_ntp_servers=127.4.177.20:41343
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false with env {}
W20260812 06:37:29.174088 12011 flags.cc:432] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260812 06:37:29.174422 12011 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:37:29.174503 12011 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:37:29.174580 12011 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 06:37:29.178000 12011 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:37:29.178153 12011 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.177.1
I20260812 06:37:29.181900 12011 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestReplaceChangeConfigOperation.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.177.20:41343
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestReplaceChangeConfigOperation.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestReplaceChangeConfigOperation.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.177.1:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestReplaceChangeConfigOperation.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--webserver_interface=127.4.177.1
--webserver_port=0
--tserver_master_addrs=127.4.177.62:32865
--never_fsync=true
--heap_profile_path=/tmp/kudu.12011
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.177.1

Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 06:37:29.183018 12011 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:37:29.184094 12011 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 06:37:29.190294 12017 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:37:29.190409 12018 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:37:29.190284 12024 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:37:29.190580 12011 server_base.cc:1061] running on GCE node
I20260812 06:37:29.191035 12011 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:37:29.191619 12011 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:37:29.192791 12011 hybrid_clock.cc:648] HybridClock initialized: now 1786516649192761 us; error 39 us; skew 500 ppm
I20260812 06:37:29.194783 12011 webserver.cc:533] Webserver started at http://127.4.177.1:33887/ using document root <none> and password file <none>
I20260812 06:37:29.195459 12011 fs_manager.cc:362] Metadata directory not provided
I20260812 06:37:29.195542 12011 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:37:29.195741 12011 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:37:29.197618 12011 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestReplaceChangeConfigOperation.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-0/data/instance:
uuid: "3b2b870051e548b7978192a6f3e375b9"
format_stamp: "Formatted at 2026-08-12 06:37:29 on dist-test-slave-jztv"
I20260812 06:37:29.198161 12011 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestReplaceChangeConfigOperation.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-0/wal/instance:
uuid: "3b2b870051e548b7978192a6f3e375b9"
format_stamp: "Formatted at 2026-08-12 06:37:29 on dist-test-slave-jztv"
I20260812 06:37:29.202350 12011 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.003s	sys 0.001s
I20260812 06:37:29.204818 12033 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 06:37:29.205823 12011 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260812 06:37:29.205957 12011 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestReplaceChangeConfigOperation.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-0/data,/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestReplaceChangeConfigOperation.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-0/wal
uuid: "3b2b870051e548b7978192a6f3e375b9"
format_stamp: "Formatted at 2026-08-12 06:37:29 on dist-test-slave-jztv"
I20260812 06:37:29.206099 12011 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestReplaceChangeConfigOperation.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-0/wal
metadata directory: /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestReplaceChangeConfigOperation.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-0/wal
1 data directories: /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestReplaceChangeConfigOperation.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:37:29.220038 12011 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 06:37:29.220947 12011 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:37:29.221181 12011 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:37:29.221810 12011 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260812 06:37:29.222950 12011 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:37:29.223026 12011 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260812 06:37:29.223097 12011 ts_tablet_manager.cc:616] Registered 0 tablets
I20260812 06:37:29.223141 12011 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260812 06:37:29.233587 12011 rpc_server.cc:307] RPC server started. Bound to: 127.4.177.1:39191
I20260812 06:37:29.233628 12252 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.4.177.1:39191 every 8 connection(s)
I20260812 06:37:29.234618 12011 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestReplaceChangeConfigOperation.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-0/data/info.pb
I20260812 06:37:29.243479 12253 heartbeater.cc:344] Connected to a master server at 127.4.177.62:32865
I20260812 06:37:29.243734 12253 heartbeater.cc:461] Registering TS with master...
I20260812 06:37:29.244180  4804 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskCnZu03/build/debug/bin/kudu as pid 12011
I20260812 06:37:29.244313  4804 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestReplaceChangeConfigOperation.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-0/wal/instance
I20260812 06:37:29.244269 12253 heartbeater.cc:507] Master 127.4.177.62:32865 requested a full tablet report, sending...
I20260812 06:37:29.245812 11918 ts_manager.cc:194] Registered new tserver with Master: 3b2b870051e548b7978192a6f3e375b9 (127.4.177.1:39191)
I20260812 06:37:29.246905  4804 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskCnZu03/build/debug/bin/kudu
/tmp/dist-test-taskCnZu03/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestReplaceChangeConfigOperation.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestReplaceChangeConfigOperation.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestReplaceChangeConfigOperation.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestReplaceChangeConfigOperation.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.177.2:0
--local_ip_for_outbound_sockets=127.4.177.2
--webserver_interface=127.4.177.2
--webserver_port=0
--tserver_master_addrs=127.4.177.62:32865
--builtin_ntp_servers=127.4.177.20:41343
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false with env {}
I20260812 06:37:29.247381 11918 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.4.177.1:59441
W20260812 06:37:29.366873 12258 flags.cc:432] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260812 06:37:29.367084 12258 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:37:29.367105 12258 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:37:29.367205 12258 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 06:37:29.370493 12258 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:37:29.370595 12258 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.177.2
I20260812 06:37:29.374445 12258 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestReplaceChangeConfigOperation.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-1/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.177.20:41343
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestReplaceChangeConfigOperation.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestReplaceChangeConfigOperation.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.177.2:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestReplaceChangeConfigOperation.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--webserver_interface=127.4.177.2
--webserver_port=0
--tserver_master_addrs=127.4.177.62:32865
--never_fsync=true
--heap_profile_path=/tmp/kudu.12258
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.177.2

Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 06:37:29.375582 12258 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:37:29.376701 12258 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 06:37:29.383574 12271 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:37:29.383634 12258 server_base.cc:1061] running on GCE node
W20260812 06:37:29.383540 12268 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:37:29.383536 12265 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:37:29.384270 12258 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:37:29.384943 12258 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:37:29.386158 12258 hybrid_clock.cc:648] HybridClock initialized: now 1786516649386155 us; error 37 us; skew 500 ppm
I20260812 06:37:29.387969 12258 webserver.cc:533] Webserver started at http://127.4.177.2:42811/ using document root <none> and password file <none>
I20260812 06:37:29.388569 12258 fs_manager.cc:362] Metadata directory not provided
I20260812 06:37:29.388659 12258 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:37:29.388947 12258 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:37:29.390679 12258 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestReplaceChangeConfigOperation.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-1/data/instance:
uuid: "72001361a79346a7a6a5c755bdc62785"
format_stamp: "Formatted at 2026-08-12 06:37:29 on dist-test-slave-jztv"
I20260812 06:37:29.391122 12258 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestReplaceChangeConfigOperation.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-1/wal/instance:
uuid: "72001361a79346a7a6a5c755bdc62785"
format_stamp: "Formatted at 2026-08-12 06:37:29 on dist-test-slave-jztv"
I20260812 06:37:29.394402 12258 fs_manager.cc:696] Time spent creating directory manager: real 0.003s	user 0.003s	sys 0.001s
I20260812 06:37:29.396600 12280 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 06:37:29.397619 12258 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260812 06:37:29.397755 12258 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestReplaceChangeConfigOperation.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-1/data,/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestReplaceChangeConfigOperation.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-1/wal
uuid: "72001361a79346a7a6a5c755bdc62785"
format_stamp: "Formatted at 2026-08-12 06:37:29 on dist-test-slave-jztv"
I20260812 06:37:29.397863 12258 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestReplaceChangeConfigOperation.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-1/wal
metadata directory: /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestReplaceChangeConfigOperation.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-1/wal
1 data directories: /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestReplaceChangeConfigOperation.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:37:29.412390 12258 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 06:37:29.413100 12258 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:37:29.413311 12258 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:37:29.413892 12258 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260812 06:37:29.414963 12258 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:37:29.415035 12258 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260812 06:37:29.415107 12258 ts_tablet_manager.cc:616] Registered 0 tablets
I20260812 06:37:29.415159 12258 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260812 06:37:29.424782 12258 rpc_server.cc:307] RPC server started. Bound to: 127.4.177.2:45649
I20260812 06:37:29.424844 12487 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.4.177.2:45649 every 8 connection(s)
I20260812 06:37:29.425772 12258 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestReplaceChangeConfigOperation.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-1/data/info.pb
I20260812 06:37:29.434602 12488 heartbeater.cc:344] Connected to a master server at 127.4.177.62:32865
I20260812 06:37:29.434862 12488 heartbeater.cc:461] Registering TS with master...
I20260812 06:37:29.435117  4804 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskCnZu03/build/debug/bin/kudu as pid 12258
I20260812 06:37:29.435251  4804 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestReplaceChangeConfigOperation.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-1/wal/instance
I20260812 06:37:29.435480 12488 heartbeater.cc:507] Master 127.4.177.62:32865 requested a full tablet report, sending...
I20260812 06:37:29.436486 11914 ts_manager.cc:194] Registered new tserver with Master: 72001361a79346a7a6a5c755bdc62785 (127.4.177.2:45649)
I20260812 06:37:29.437125 11914 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.4.177.2:55695
I20260812 06:37:29.437525  4804 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskCnZu03/build/debug/bin/kudu
/tmp/dist-test-taskCnZu03/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestReplaceChangeConfigOperation.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestReplaceChangeConfigOperation.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestReplaceChangeConfigOperation.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestReplaceChangeConfigOperation.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.177.3:0
--local_ip_for_outbound_sockets=127.4.177.3
--webserver_interface=127.4.177.3
--webserver_port=0
--tserver_master_addrs=127.4.177.62:32865
--builtin_ntp_servers=127.4.177.20:41343
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false with env {}
W20260812 06:37:29.543493 12502 flags.cc:432] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260812 06:37:29.543731 12502 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:37:29.543756 12502 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:37:29.543793 12502 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 06:37:29.547034 12502 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:37:29.547139 12502 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.177.3
I20260812 06:37:29.550936 12502 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestReplaceChangeConfigOperation.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-2/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.177.20:41343
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestReplaceChangeConfigOperation.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestReplaceChangeConfigOperation.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.177.3:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestReplaceChangeConfigOperation.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--webserver_interface=127.4.177.3
--webserver_port=0
--tserver_master_addrs=127.4.177.62:32865
--never_fsync=true
--heap_profile_path=/tmp/kudu.12502
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.177.3

Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 06:37:29.551959 12502 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:37:29.553030 12502 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 06:37:29.559427 12513 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:37:29.559443 12514 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:37:29.559512 12520 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:37:29.559818 12502 server_base.cc:1061] running on GCE node
I20260812 06:37:29.560173 12502 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:37:29.560765 12502 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:37:29.561964 12502 hybrid_clock.cc:648] HybridClock initialized: now 1786516649561941 us; error 34 us; skew 500 ppm
I20260812 06:37:29.563769 12502 webserver.cc:533] Webserver started at http://127.4.177.3:32885/ using document root <none> and password file <none>
I20260812 06:37:29.564327 12502 fs_manager.cc:362] Metadata directory not provided
I20260812 06:37:29.564426 12502 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:37:29.564652 12502 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:37:29.566140 12502 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestReplaceChangeConfigOperation.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-2/data/instance:
uuid: "4c9bb25173564fe383ecc4ee6d4be146"
format_stamp: "Formatted at 2026-08-12 06:37:29 on dist-test-slave-jztv"
I20260812 06:37:29.566702 12502 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestReplaceChangeConfigOperation.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-2/wal/instance:
uuid: "4c9bb25173564fe383ecc4ee6d4be146"
format_stamp: "Formatted at 2026-08-12 06:37:29 on dist-test-slave-jztv"
I20260812 06:37:29.569830 12502 fs_manager.cc:696] Time spent creating directory manager: real 0.003s	user 0.003s	sys 0.000s
I20260812 06:37:29.572342 12532 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 06:37:29.573235 12502 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260812 06:37:29.573370 12502 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestReplaceChangeConfigOperation.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-2/data,/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestReplaceChangeConfigOperation.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-2/wal
uuid: "4c9bb25173564fe383ecc4ee6d4be146"
format_stamp: "Formatted at 2026-08-12 06:37:29 on dist-test-slave-jztv"
I20260812 06:37:29.573475 12502 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestReplaceChangeConfigOperation.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-2/wal
metadata directory: /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestReplaceChangeConfigOperation.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-2/wal
1 data directories: /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestReplaceChangeConfigOperation.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:37:29.583511 12502 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 06:37:29.584112 12502 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:37:29.584299 12502 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:37:29.584831 12502 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260812 06:37:29.585793 12502 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:37:29.585867 12502 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260812 06:37:29.585932 12502 ts_tablet_manager.cc:616] Registered 0 tablets
I20260812 06:37:29.585983 12502 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260812 06:37:29.595609 12502 rpc_server.cc:307] RPC server started. Bound to: 127.4.177.3:33169
I20260812 06:37:29.595638 12722 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.4.177.3:33169 every 8 connection(s)
I20260812 06:37:29.596524 12502 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestReplaceChangeConfigOperation.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-2/data/info.pb
I20260812 06:37:29.604377  4804 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskCnZu03/build/debug/bin/kudu as pid 12502
I20260812 06:37:29.604550  4804 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestReplaceChangeConfigOperation.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-2/wal/instance
I20260812 06:37:29.605151 12723 heartbeater.cc:344] Connected to a master server at 127.4.177.62:32865
I20260812 06:37:29.605388 12723 heartbeater.cc:461] Registering TS with master...
I20260812 06:37:29.605947 12723 heartbeater.cc:507] Master 127.4.177.62:32865 requested a full tablet report, sending...
I20260812 06:37:29.607122 11918 ts_manager.cc:194] Registered new tserver with Master: 4c9bb25173564fe383ecc4ee6d4be146 (127.4.177.3:33169)
I20260812 06:37:29.607756 11918 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.4.177.3:49771
I20260812 06:37:29.619190  4804 external_mini_cluster.cc:955] 3 TS(s) registered with all masters
I20260812 06:37:29.628217 11918 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47600:
name: "TestTable"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
owner: "alice"
W20260812 06:37:29.630954 11918 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table TestTable in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 06:37:29.648342 12605 tablet_service.cc:1511] Processing CreateTablet for tablet 328795a7f87644e0837e1d1519e1454d (DEFAULT_TABLE table=TestTable [id=95751e2dde5243bba77e4c6a2a99358f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 06:37:29.648548 12134 tablet_service.cc:1511] Processing CreateTablet for tablet 328795a7f87644e0837e1d1519e1454d (DEFAULT_TABLE table=TestTable [id=95751e2dde5243bba77e4c6a2a99358f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 06:37:29.648590 12376 tablet_service.cc:1511] Processing CreateTablet for tablet 328795a7f87644e0837e1d1519e1454d (DEFAULT_TABLE table=TestTable [id=95751e2dde5243bba77e4c6a2a99358f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 06:37:29.649390 12134 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 328795a7f87644e0837e1d1519e1454d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:37:29.649412 12376 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 328795a7f87644e0837e1d1519e1454d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:37:29.649412 12605 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 328795a7f87644e0837e1d1519e1454d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:37:29.654500 12756 tablet_bootstrap.cc:492] T 328795a7f87644e0837e1d1519e1454d P 3b2b870051e548b7978192a6f3e375b9: Bootstrap starting.
I20260812 06:37:29.655712 12760 tablet_bootstrap.cc:492] T 328795a7f87644e0837e1d1519e1454d P 72001361a79346a7a6a5c755bdc62785: Bootstrap starting.
I20260812 06:37:29.657117 12756 tablet_bootstrap.cc:654] T 328795a7f87644e0837e1d1519e1454d P 3b2b870051e548b7978192a6f3e375b9: Neither blocks nor log segments found. Creating new log.
I20260812 06:37:29.657593 12760 tablet_bootstrap.cc:654] T 328795a7f87644e0837e1d1519e1454d P 72001361a79346a7a6a5c755bdc62785: Neither blocks nor log segments found. Creating new log.
I20260812 06:37:29.657873 12756 log.cc:826] T 328795a7f87644e0837e1d1519e1454d P 3b2b870051e548b7978192a6f3e375b9: Log is configured to *not* fsync() on all Append() calls
I20260812 06:37:29.658334 12760 log.cc:826] T 328795a7f87644e0837e1d1519e1454d P 72001361a79346a7a6a5c755bdc62785: Log is configured to *not* fsync() on all Append() calls
I20260812 06:37:29.658592 12762 tablet_bootstrap.cc:492] T 328795a7f87644e0837e1d1519e1454d P 4c9bb25173564fe383ecc4ee6d4be146: Bootstrap starting.
I20260812 06:37:29.659749 12756 tablet_bootstrap.cc:492] T 328795a7f87644e0837e1d1519e1454d P 3b2b870051e548b7978192a6f3e375b9: No bootstrap required, opened a new log
I20260812 06:37:29.659782 12760 tablet_bootstrap.cc:492] T 328795a7f87644e0837e1d1519e1454d P 72001361a79346a7a6a5c755bdc62785: No bootstrap required, opened a new log
I20260812 06:37:29.659934 12756 ts_tablet_manager.cc:1403] T 328795a7f87644e0837e1d1519e1454d P 3b2b870051e548b7978192a6f3e375b9: Time spent bootstrapping tablet: real 0.006s	user 0.004s	sys 0.000s
I20260812 06:37:29.659968 12760 ts_tablet_manager.cc:1403] T 328795a7f87644e0837e1d1519e1454d P 72001361a79346a7a6a5c755bdc62785: Time spent bootstrapping tablet: real 0.004s	user 0.000s	sys 0.003s
I20260812 06:37:29.660715 12762 tablet_bootstrap.cc:654] T 328795a7f87644e0837e1d1519e1454d P 4c9bb25173564fe383ecc4ee6d4be146: Neither blocks nor log segments found. Creating new log.
I20260812 06:37:29.661501 12762 log.cc:826] T 328795a7f87644e0837e1d1519e1454d P 4c9bb25173564fe383ecc4ee6d4be146: Log is configured to *not* fsync() on all Append() calls
I20260812 06:37:29.662809 12760 raft_consensus.cc:359] T 328795a7f87644e0837e1d1519e1454d P 72001361a79346a7a6a5c755bdc62785 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c9bb25173564fe383ecc4ee6d4be146" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 33169 } } peers { permanent_uuid: "72001361a79346a7a6a5c755bdc62785" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 45649 } } peers { permanent_uuid: "3b2b870051e548b7978192a6f3e375b9" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 39191 } }
I20260812 06:37:29.663017 12760 raft_consensus.cc:740] T 328795a7f87644e0837e1d1519e1454d P 72001361a79346a7a6a5c755bdc62785 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 72001361a79346a7a6a5c755bdc62785, State: Initialized, Role: FOLLOWER
I20260812 06:37:29.663000 12756 raft_consensus.cc:359] T 328795a7f87644e0837e1d1519e1454d P 3b2b870051e548b7978192a6f3e375b9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c9bb25173564fe383ecc4ee6d4be146" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 33169 } } peers { permanent_uuid: "72001361a79346a7a6a5c755bdc62785" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 45649 } } peers { permanent_uuid: "3b2b870051e548b7978192a6f3e375b9" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 39191 } }
I20260812 06:37:29.663215 12756 raft_consensus.cc:740] T 328795a7f87644e0837e1d1519e1454d P 3b2b870051e548b7978192a6f3e375b9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b2b870051e548b7978192a6f3e375b9, State: Initialized, Role: FOLLOWER
I20260812 06:37:29.663400 12760 consensus_queue.cc:260] T 328795a7f87644e0837e1d1519e1454d P 72001361a79346a7a6a5c755bdc62785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c9bb25173564fe383ecc4ee6d4be146" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 33169 } } peers { permanent_uuid: "72001361a79346a7a6a5c755bdc62785" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 45649 } } peers { permanent_uuid: "3b2b870051e548b7978192a6f3e375b9" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 39191 } }
I20260812 06:37:29.663506 12756 consensus_queue.cc:260] T 328795a7f87644e0837e1d1519e1454d P 3b2b870051e548b7978192a6f3e375b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c9bb25173564fe383ecc4ee6d4be146" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 33169 } } peers { permanent_uuid: "72001361a79346a7a6a5c755bdc62785" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 45649 } } peers { permanent_uuid: "3b2b870051e548b7978192a6f3e375b9" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 39191 } }
I20260812 06:37:29.663596 12762 tablet_bootstrap.cc:492] T 328795a7f87644e0837e1d1519e1454d P 4c9bb25173564fe383ecc4ee6d4be146: No bootstrap required, opened a new log
I20260812 06:37:29.663777 12762 ts_tablet_manager.cc:1403] T 328795a7f87644e0837e1d1519e1454d P 4c9bb25173564fe383ecc4ee6d4be146: Time spent bootstrapping tablet: real 0.005s	user 0.004s	sys 0.000s
I20260812 06:37:29.664203 12253 heartbeater.cc:499] Master 127.4.177.62:32865 was elected leader, sending a full tablet report...
I20260812 06:37:29.664280 12760 ts_tablet_manager.cc:1434] T 328795a7f87644e0837e1d1519e1454d P 72001361a79346a7a6a5c755bdc62785: Time spent starting tablet: real 0.004s	user 0.000s	sys 0.005s
I20260812 06:37:29.664325 12756 ts_tablet_manager.cc:1434] T 328795a7f87644e0837e1d1519e1454d P 3b2b870051e548b7978192a6f3e375b9: Time spent starting tablet: real 0.004s	user 0.005s	sys 0.000s
I20260812 06:37:29.664635 12488 heartbeater.cc:499] Master 127.4.177.62:32865 was elected leader, sending a full tablet report...
I20260812 06:37:29.667340 12762 raft_consensus.cc:359] T 328795a7f87644e0837e1d1519e1454d P 4c9bb25173564fe383ecc4ee6d4be146 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c9bb25173564fe383ecc4ee6d4be146" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 33169 } } peers { permanent_uuid: "72001361a79346a7a6a5c755bdc62785" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 45649 } } peers { permanent_uuid: "3b2b870051e548b7978192a6f3e375b9" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 39191 } }
I20260812 06:37:29.667501 12762 raft_consensus.cc:740] T 328795a7f87644e0837e1d1519e1454d P 4c9bb25173564fe383ecc4ee6d4be146 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4c9bb25173564fe383ecc4ee6d4be146, State: Initialized, Role: FOLLOWER
I20260812 06:37:29.667846 12762 consensus_queue.cc:260] T 328795a7f87644e0837e1d1519e1454d P 4c9bb25173564fe383ecc4ee6d4be146 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c9bb25173564fe383ecc4ee6d4be146" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 33169 } } peers { permanent_uuid: "72001361a79346a7a6a5c755bdc62785" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 45649 } } peers { permanent_uuid: "3b2b870051e548b7978192a6f3e375b9" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 39191 } }
I20260812 06:37:29.668748 12762 ts_tablet_manager.cc:1434] T 328795a7f87644e0837e1d1519e1454d P 4c9bb25173564fe383ecc4ee6d4be146: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260812 06:37:29.668881 12723 heartbeater.cc:499] Master 127.4.177.62:32865 was elected leader, sending a full tablet report...
I20260812 06:37:29.672952  4804 external_mini_cluster.cc:955] 3 TS(s) registered with all masters
I20260812 06:37:29.674441  4804 ts_itest-base.cc:246] Waiting for 1 tablets on tserver 3b2b870051e548b7978192a6f3e375b9 to finish bootstrapping
W20260812 06:37:29.676713 12494 tablet.cc:2406] T 328795a7f87644e0837e1d1519e1454d P 72001361a79346a7a6a5c755bdc62785: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 06:37:29.680984  4804 ts_itest-base.cc:246] Waiting for 1 tablets on tserver 72001361a79346a7a6a5c755bdc62785 to finish bootstrapping
I20260812 06:37:29.686785  4804 ts_itest-base.cc:246] Waiting for 1 tablets on tserver 4c9bb25173564fe383ecc4ee6d4be146 to finish bootstrapping
I20260812 06:37:29.695870 12169 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "328795a7f87644e0837e1d1519e1454d"
dest_uuid: "3b2b870051e548b7978192a6f3e375b9"
 from {username='slave'} at 127.0.0.1:60666
I20260812 06:37:29.696149 12169 raft_consensus.cc:493] T 328795a7f87644e0837e1d1519e1454d P 3b2b870051e548b7978192a6f3e375b9 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260812 06:37:29.696288 12169 raft_consensus.cc:3060] T 328795a7f87644e0837e1d1519e1454d P 3b2b870051e548b7978192a6f3e375b9 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:37:29.697279 12169 raft_consensus.cc:515] T 328795a7f87644e0837e1d1519e1454d P 3b2b870051e548b7978192a6f3e375b9 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c9bb25173564fe383ecc4ee6d4be146" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 33169 } } peers { permanent_uuid: "72001361a79346a7a6a5c755bdc62785" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 45649 } } peers { permanent_uuid: "3b2b870051e548b7978192a6f3e375b9" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 39191 } }
I20260812 06:37:29.698443 12169 leader_election.cc:290] T 328795a7f87644e0837e1d1519e1454d P 3b2b870051e548b7978192a6f3e375b9 [CANDIDATE]: Term 1 election: Requested vote from peers 4c9bb25173564fe383ecc4ee6d4be146 (127.4.177.3:33169), 72001361a79346a7a6a5c755bdc62785 (127.4.177.2:45649)
I20260812 06:37:29.702860 12407 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "328795a7f87644e0837e1d1519e1454d" candidate_uuid: "3b2b870051e548b7978192a6f3e375b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "72001361a79346a7a6a5c755bdc62785"
I20260812 06:37:29.703083 12407 raft_consensus.cc:3060] T 328795a7f87644e0837e1d1519e1454d P 72001361a79346a7a6a5c755bdc62785 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:37:29.704044 12638 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "328795a7f87644e0837e1d1519e1454d" candidate_uuid: "3b2b870051e548b7978192a6f3e375b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "4c9bb25173564fe383ecc4ee6d4be146"
I20260812 06:37:29.704274 12638 raft_consensus.cc:3060] T 328795a7f87644e0837e1d1519e1454d P 4c9bb25173564fe383ecc4ee6d4be146 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:37:29.704442 12407 raft_consensus.cc:2468] T 328795a7f87644e0837e1d1519e1454d P 72001361a79346a7a6a5c755bdc62785 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b2b870051e548b7978192a6f3e375b9 in term 1.
I20260812 06:37:29.705257 12638 raft_consensus.cc:2468] T 328795a7f87644e0837e1d1519e1454d P 4c9bb25173564fe383ecc4ee6d4be146 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b2b870051e548b7978192a6f3e375b9 in term 1.
I20260812 06:37:29.705705 12041 leader_election.cc:304] T 328795a7f87644e0837e1d1519e1454d P 3b2b870051e548b7978192a6f3e375b9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b2b870051e548b7978192a6f3e375b9, 4c9bb25173564fe383ecc4ee6d4be146; no voters: 
I20260812 06:37:29.706041 12768 raft_consensus.cc:2804] T 328795a7f87644e0837e1d1519e1454d P 3b2b870051e548b7978192a6f3e375b9 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 06:37:29.706280 12768 raft_consensus.cc:697] T 328795a7f87644e0837e1d1519e1454d P 3b2b870051e548b7978192a6f3e375b9 [term 1 LEADER]: Becoming Leader. State: Replica: 3b2b870051e548b7978192a6f3e375b9, State: Running, Role: LEADER
I20260812 06:37:29.706624 12768 consensus_queue.cc:237] T 328795a7f87644e0837e1d1519e1454d P 3b2b870051e548b7978192a6f3e375b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c9bb25173564fe383ecc4ee6d4be146" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 33169 } } peers { permanent_uuid: "72001361a79346a7a6a5c755bdc62785" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 45649 } } peers { permanent_uuid: "3b2b870051e548b7978192a6f3e375b9" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 39191 } }
I20260812 06:37:29.710026 11913 catalog_manager.cc:5719] T 328795a7f87644e0837e1d1519e1454d P 3b2b870051e548b7978192a6f3e375b9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3b2b870051e548b7978192a6f3e375b9 (127.4.177.1). New cstate: current_term: 1 leader_uuid: "3b2b870051e548b7978192a6f3e375b9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c9bb25173564fe383ecc4ee6d4be146" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 33169 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "72001361a79346a7a6a5c755bdc62785" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 45649 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3b2b870051e548b7978192a6f3e375b9" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 39191 } health_report { overall_health: HEALTHY } } }
I20260812 06:37:29.710366  4804 cluster_itest_util.cc:258] Not converged past 1 yet: 0.0 0.0 0.0
W20260812 06:37:29.735832 12254 tablet.cc:2406] T 328795a7f87644e0837e1d1519e1454d P 3b2b870051e548b7978192a6f3e375b9: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 06:37:29.813241  4804 cluster_itest_util.cc:258] Not converged past 1 yet: 1.1 0.0 0.0
W20260812 06:37:29.847503 12725 tablet.cc:2406] T 328795a7f87644e0837e1d1519e1454d P 4c9bb25173564fe383ecc4ee6d4be146: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 06:37:30.016315  4804 cluster_itest_util.cc:258] Not converged past 1 yet: 1.1 0.0 0.0
I20260812 06:37:30.092864 12768 consensus_queue.cc:1048] T 328795a7f87644e0837e1d1519e1454d P 3b2b870051e548b7978192a6f3e375b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4c9bb25173564fe383ecc4ee6d4be146" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 33169 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 06:37:30.093119 12782 consensus_queue.cc:1048] T 328795a7f87644e0837e1d1519e1454d P 3b2b870051e548b7978192a6f3e375b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "72001361a79346a7a6a5c755bdc62785" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 45649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 06:37:30.319445  4804 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskCnZu03/build/debug/bin/kudu with pid 12258
I20260812 06:37:30.335310  4804 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskCnZu03/build/debug/bin/kudu with pid 12502
I20260812 06:37:30.351820 12169 consensus_queue.cc:237] T 328795a7f87644e0837e1d1519e1454d P 3b2b870051e548b7978192a6f3e375b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4c9bb25173564fe383ecc4ee6d4be146" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 33169 } } peers { permanent_uuid: "3b2b870051e548b7978192a6f3e375b9" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 39191 } }
W20260812 06:37:30.353272 12041 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.4.177.3:33169: connect: Connection refused (error 111)
W20260812 06:37:30.354100 12041 consensus_peers.cc:597] T 328795a7f87644e0837e1d1519e1454d P 3b2b870051e548b7978192a6f3e375b9 -> Peer 4c9bb25173564fe383ecc4ee6d4be146 (127.4.177.3:33169): Couldn't send request to peer 4c9bb25173564fe383ecc4ee6d4be146. Status: Network error: Client connection negotiation failed: client connection to 127.4.177.3:33169: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260812 06:37:31.363044  4804 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskCnZu03/build/debug/bin/kudu
/tmp/dist-test-taskCnZu03/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestReplaceChangeConfigOperation.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestReplaceChangeConfigOperation.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestReplaceChangeConfigOperation.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestReplaceChangeConfigOperation.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.177.2:45649
--local_ip_for_outbound_sockets=127.4.177.2
--tserver_master_addrs=127.4.177.62:32865
--webserver_port=42811
--webserver_interface=127.4.177.2
--builtin_ntp_servers=127.4.177.20:41343
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false with env {}
W20260812 06:37:31.467428 12802 flags.cc:432] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260812 06:37:31.467711 12802 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:37:31.467767 12802 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:37:31.467839 12802 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 06:37:31.471086 12802 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:37:31.471261 12802 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.177.2
I20260812 06:37:31.474956 12802 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestReplaceChangeConfigOperation.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-1/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.177.20:41343
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestReplaceChangeConfigOperation.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestReplaceChangeConfigOperation.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.177.2:45649
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestReplaceChangeConfigOperation.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--webserver_interface=127.4.177.2
--webserver_port=42811
--tserver_master_addrs=127.4.177.62:32865
--never_fsync=true
--heap_profile_path=/tmp/kudu.12802
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.177.2

Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 06:37:31.476049 12802 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:37:31.477231 12802 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 06:37:31.483760 12814 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:37:31.483891 12819 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:37:31.483920 12816 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:37:31.484071 12802 server_base.cc:1061] running on GCE node
I20260812 06:37:31.484468 12802 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:37:31.485061 12802 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:37:31.486289 12802 hybrid_clock.cc:648] HybridClock initialized: now 1786516651486277 us; error 28 us; skew 500 ppm
I20260812 06:37:31.488127 12802 webserver.cc:533] Webserver started at http://127.4.177.2:42811/ using document root <none> and password file <none>
I20260812 06:37:31.488716 12802 fs_manager.cc:362] Metadata directory not provided
I20260812 06:37:31.488831 12802 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:37:31.492259 12802 fs_manager.cc:714] Time spent opening directory manager: real 0.002s	user 0.003s	sys 0.000s
I20260812 06:37:31.494184 12831 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 06:37:31.495163 12802 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260812 06:37:31.495301 12802 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestReplaceChangeConfigOperation.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-1/data,/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestReplaceChangeConfigOperation.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-1/wal
uuid: "72001361a79346a7a6a5c755bdc62785"
format_stamp: "Formatted at 2026-08-12 06:37:29 on dist-test-slave-jztv"
I20260812 06:37:31.495703 12802 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestReplaceChangeConfigOperation.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-1/wal
metadata directory: /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestReplaceChangeConfigOperation.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-1/wal
1 data directories: /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestReplaceChangeConfigOperation.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:37:31.505640 12802 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 06:37:31.506310 12802 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:37:31.506510 12802 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:37:31.507053 12802 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260812 06:37:31.508301 12842 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
I20260812 06:37:31.511521 12802 ts_tablet_manager.cc:585] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260812 06:37:31.511615 12802 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.004s	user 0.000s	sys 0.000s
I20260812 06:37:31.511718 12802 ts_tablet_manager.cc:600] Registering tablets (0/1 complete)
I20260812 06:37:31.513106 12802 ts_tablet_manager.cc:616] Registered 1 tablets
I20260812 06:37:31.513177 12802 ts_tablet_manager.cc:595] Time spent register tablets: real 0.001s	user 0.000s	sys 0.000s
I20260812 06:37:31.513458 12842 tablet_bootstrap.cc:492] T 328795a7f87644e0837e1d1519e1454d P 72001361a79346a7a6a5c755bdc62785: Bootstrap starting.
I20260812 06:37:31.525535 12802 rpc_server.cc:307] RPC server started. Bound to: 127.4.177.2:45649
I20260812 06:37:31.525557 13026 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.4.177.2:45649 every 8 connection(s)
I20260812 06:37:31.526757 12802 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestReplaceChangeConfigOperation.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-1/data/info.pb
I20260812 06:37:31.529043 12842 log.cc:826] T 328795a7f87644e0837e1d1519e1454d P 72001361a79346a7a6a5c755bdc62785: Log is configured to *not* fsync() on all Append() calls
I20260812 06:37:31.530270  4804 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskCnZu03/build/debug/bin/kudu as pid 12802
I20260812 06:37:31.530879  4804 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskCnZu03/build/debug/bin/kudu
/tmp/dist-test-taskCnZu03/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestReplaceChangeConfigOperation.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestReplaceChangeConfigOperation.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestReplaceChangeConfigOperation.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestReplaceChangeConfigOperation.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.177.3:33169
--local_ip_for_outbound_sockets=127.4.177.3
--tserver_master_addrs=127.4.177.62:32865
--webserver_port=32885
--webserver_interface=127.4.177.3
--builtin_ntp_servers=127.4.177.20:41343
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false with env {}
I20260812 06:37:31.531899 12842 tablet_bootstrap.cc:492] T 328795a7f87644e0837e1d1519e1454d P 72001361a79346a7a6a5c755bdc62785: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 06:37:31.532286 12842 tablet_bootstrap.cc:492] T 328795a7f87644e0837e1d1519e1454d P 72001361a79346a7a6a5c755bdc62785: Bootstrap complete.
I20260812 06:37:31.533668 12842 ts_tablet_manager.cc:1403] T 328795a7f87644e0837e1d1519e1454d P 72001361a79346a7a6a5c755bdc62785: Time spent bootstrapping tablet: real 0.020s	user 0.011s	sys 0.008s
I20260812 06:37:31.536352 12842 raft_consensus.cc:359] T 328795a7f87644e0837e1d1519e1454d P 72001361a79346a7a6a5c755bdc62785 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c9bb25173564fe383ecc4ee6d4be146" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 33169 } } peers { permanent_uuid: "72001361a79346a7a6a5c755bdc62785" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 45649 } } peers { permanent_uuid: "3b2b870051e548b7978192a6f3e375b9" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 39191 } }
I20260812 06:37:31.536702 12842 raft_consensus.cc:740] T 328795a7f87644e0837e1d1519e1454d P 72001361a79346a7a6a5c755bdc62785 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 72001361a79346a7a6a5c755bdc62785, State: Initialized, Role: FOLLOWER
I20260812 06:37:31.537024 12842 consensus_queue.cc:260] T 328795a7f87644e0837e1d1519e1454d P 72001361a79346a7a6a5c755bdc62785 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c9bb25173564fe383ecc4ee6d4be146" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 33169 } } peers { permanent_uuid: "72001361a79346a7a6a5c755bdc62785" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 45649 } } peers { permanent_uuid: "3b2b870051e548b7978192a6f3e375b9" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 39191 } }
I20260812 06:37:31.537842 12842 ts_tablet_manager.cc:1434] T 328795a7f87644e0837e1d1519e1454d P 72001361a79346a7a6a5c755bdc62785: Time spent starting tablet: real 0.004s	user 0.005s	sys 0.000s
I20260812 06:37:31.540515 13027 heartbeater.cc:344] Connected to a master server at 127.4.177.62:32865
I20260812 06:37:31.540781 13027 heartbeater.cc:461] Registering TS with master...
I20260812 06:37:31.541323 13027 heartbeater.cc:507] Master 127.4.177.62:32865 requested a full tablet report, sending...
I20260812 06:37:31.543249 11912 ts_manager.cc:194] Re-registered known tserver with Master: 72001361a79346a7a6a5c755bdc62785 (127.4.177.2:45649)
I20260812 06:37:31.544071 11912 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.4.177.2:46155
I20260812 06:37:31.545706 13027 heartbeater.cc:499] Master 127.4.177.62:32865 was elected leader, sending a full tablet report...
W20260812 06:37:31.639689 13033 flags.cc:432] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260812 06:37:31.639991 13033 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:37:31.640064 13033 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:37:31.640151 13033 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 06:37:31.643529 13033 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:37:31.643707 13033 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.177.3
I20260812 06:37:31.647591 13033 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestReplaceChangeConfigOperation.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-2/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.177.20:41343
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestReplaceChangeConfigOperation.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestReplaceChangeConfigOperation.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.177.3:33169
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestReplaceChangeConfigOperation.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--webserver_interface=127.4.177.3
--webserver_port=32885
--tserver_master_addrs=127.4.177.62:32865
--never_fsync=true
--heap_profile_path=/tmp/kudu.13033
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.177.3

Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 06:37:31.648792 13033 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:37:31.649953 13033 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 06:37:31.656536 13044 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:37:31.656538 13047 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:37:31.656538 13045 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:37:31.656726 13033 server_base.cc:1061] running on GCE node
I20260812 06:37:31.657276 13033 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:37:31.657812 13033 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:37:31.658972 13033 hybrid_clock.cc:648] HybridClock initialized: now 1786516651658944 us; error 38 us; skew 500 ppm
I20260812 06:37:31.660791 13033 webserver.cc:533] Webserver started at http://127.4.177.3:32885/ using document root <none> and password file <none>
I20260812 06:37:31.661370 13033 fs_manager.cc:362] Metadata directory not provided
I20260812 06:37:31.661464 13033 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:37:31.664780 13033 fs_manager.cc:714] Time spent opening directory manager: real 0.002s	user 0.002s	sys 0.000s
I20260812 06:37:31.666673 13059 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 06:37:31.667608 13033 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260812 06:37:31.667749 13033 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestReplaceChangeConfigOperation.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-2/data,/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestReplaceChangeConfigOperation.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-2/wal
uuid: "4c9bb25173564fe383ecc4ee6d4be146"
format_stamp: "Formatted at 2026-08-12 06:37:29 on dist-test-slave-jztv"
I20260812 06:37:31.668144 13033 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestReplaceChangeConfigOperation.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-2/wal
metadata directory: /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestReplaceChangeConfigOperation.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-2/wal
1 data directories: /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestReplaceChangeConfigOperation.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:37:31.686399 13033 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 06:37:31.687112 13033 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:37:31.687305 13033 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:37:31.687879 13033 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260812 06:37:31.689132 13070 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
I20260812 06:37:31.692384 13033 ts_tablet_manager.cc:585] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260812 06:37:31.692464 13033 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.004s	user 0.000s	sys 0.000s
I20260812 06:37:31.692561 13033 ts_tablet_manager.cc:600] Registering tablets (0/1 complete)
I20260812 06:37:31.693953 13033 ts_tablet_manager.cc:616] Registered 1 tablets
I20260812 06:37:31.694026 13033 ts_tablet_manager.cc:595] Time spent register tablets: real 0.001s	user 0.000s	sys 0.000s
I20260812 06:37:31.694383 13070 tablet_bootstrap.cc:492] T 328795a7f87644e0837e1d1519e1454d P 4c9bb25173564fe383ecc4ee6d4be146: Bootstrap starting.
I20260812 06:37:31.705856 13033 rpc_server.cc:307] RPC server started. Bound to: 127.4.177.3:33169
I20260812 06:37:31.705955 13267 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.4.177.3:33169 every 8 connection(s)
I20260812 06:37:31.707037 13033 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestReplaceChangeConfigOperation.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-2/data/info.pb
I20260812 06:37:31.709384  4804 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskCnZu03/build/debug/bin/kudu as pid 13033
I20260812 06:37:31.709728 13070 log.cc:826] T 328795a7f87644e0837e1d1519e1454d P 4c9bb25173564fe383ecc4ee6d4be146: Log is configured to *not* fsync() on all Append() calls
I20260812 06:37:31.712508 13070 tablet_bootstrap.cc:492] T 328795a7f87644e0837e1d1519e1454d P 4c9bb25173564fe383ecc4ee6d4be146: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 06:37:31.712904 13070 tablet_bootstrap.cc:492] T 328795a7f87644e0837e1d1519e1454d P 4c9bb25173564fe383ecc4ee6d4be146: Bootstrap complete.
I20260812 06:37:31.714345 13070 ts_tablet_manager.cc:1403] T 328795a7f87644e0837e1d1519e1454d P 4c9bb25173564fe383ecc4ee6d4be146: Time spent bootstrapping tablet: real 0.020s	user 0.013s	sys 0.007s
I20260812 06:37:31.717180 13269 heartbeater.cc:344] Connected to a master server at 127.4.177.62:32865
I20260812 06:37:31.717463 13269 heartbeater.cc:461] Registering TS with master...
I20260812 06:37:31.717732 13070 raft_consensus.cc:359] T 328795a7f87644e0837e1d1519e1454d P 4c9bb25173564fe383ecc4ee6d4be146 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c9bb25173564fe383ecc4ee6d4be146" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 33169 } } peers { permanent_uuid: "72001361a79346a7a6a5c755bdc62785" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 45649 } } peers { permanent_uuid: "3b2b870051e548b7978192a6f3e375b9" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 39191 } }
I20260812 06:37:31.718029 13269 heartbeater.cc:507] Master 127.4.177.62:32865 requested a full tablet report, sending...
I20260812 06:37:31.718041 13070 raft_consensus.cc:740] T 328795a7f87644e0837e1d1519e1454d P 4c9bb25173564fe383ecc4ee6d4be146 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4c9bb25173564fe383ecc4ee6d4be146, State: Initialized, Role: FOLLOWER
I20260812 06:37:31.718477 13070 consensus_queue.cc:260] T 328795a7f87644e0837e1d1519e1454d P 4c9bb25173564fe383ecc4ee6d4be146 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c9bb25173564fe383ecc4ee6d4be146" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 33169 } } peers { permanent_uuid: "72001361a79346a7a6a5c755bdc62785" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 45649 } } peers { permanent_uuid: "3b2b870051e548b7978192a6f3e375b9" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 39191 } }
I20260812 06:37:31.719440 13070 ts_tablet_manager.cc:1434] T 328795a7f87644e0837e1d1519e1454d P 4c9bb25173564fe383ecc4ee6d4be146: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.001s
I20260812 06:37:31.720367 11912 ts_manager.cc:194] Re-registered known tserver with Master: 4c9bb25173564fe383ecc4ee6d4be146 (127.4.177.3:33169)
I20260812 06:37:31.721216 11912 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.4.177.3:49859
I20260812 06:37:31.723109 13269 heartbeater.cc:499] Master 127.4.177.62:32865 was elected leader, sending a full tablet report...
I20260812 06:37:31.724591 12939 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "328795a7f87644e0837e1d1519e1454d"
dest_uuid: "72001361a79346a7a6a5c755bdc62785"
 from {username='slave'} at 127.0.0.1:41476
I20260812 06:37:31.724772 12939 raft_consensus.cc:493] T 328795a7f87644e0837e1d1519e1454d P 72001361a79346a7a6a5c755bdc62785 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260812 06:37:31.724853 12939 raft_consensus.cc:3060] T 328795a7f87644e0837e1d1519e1454d P 72001361a79346a7a6a5c755bdc62785 [term 1 FOLLOWER]: Advancing to term 2
I20260812 06:37:31.726504 12939 raft_consensus.cc:515] T 328795a7f87644e0837e1d1519e1454d P 72001361a79346a7a6a5c755bdc62785 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c9bb25173564fe383ecc4ee6d4be146" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 33169 } } peers { permanent_uuid: "72001361a79346a7a6a5c755bdc62785" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 45649 } } peers { permanent_uuid: "3b2b870051e548b7978192a6f3e375b9" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 39191 } }
I20260812 06:37:31.727520 12939 leader_election.cc:290] T 328795a7f87644e0837e1d1519e1454d P 72001361a79346a7a6a5c755bdc62785 [CANDIDATE]: Term 2 election: Requested vote from peers 4c9bb25173564fe383ecc4ee6d4be146 (127.4.177.3:33169), 3b2b870051e548b7978192a6f3e375b9 (127.4.177.1:39191)
I20260812 06:37:31.731703 13184 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "328795a7f87644e0837e1d1519e1454d" candidate_uuid: "72001361a79346a7a6a5c755bdc62785" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4c9bb25173564fe383ecc4ee6d4be146"
I20260812 06:37:31.731952 13184 raft_consensus.cc:3060] T 328795a7f87644e0837e1d1519e1454d P 4c9bb25173564fe383ecc4ee6d4be146 [term 1 FOLLOWER]: Advancing to term 2
I20260812 06:37:31.733538 13184 raft_consensus.cc:2468] T 328795a7f87644e0837e1d1519e1454d P 4c9bb25173564fe383ecc4ee6d4be146 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 72001361a79346a7a6a5c755bdc62785 in term 2.
I20260812 06:37:31.734046 12838 leader_election.cc:304] T 328795a7f87644e0837e1d1519e1454d P 72001361a79346a7a6a5c755bdc62785 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4c9bb25173564fe383ecc4ee6d4be146, 72001361a79346a7a6a5c755bdc62785; no voters: 
I20260812 06:37:31.734349 13035 raft_consensus.cc:2804] T 328795a7f87644e0837e1d1519e1454d P 72001361a79346a7a6a5c755bdc62785 [term 2 FOLLOWER]: Leader election won for term 2
I20260812 06:37:31.734596 13035 raft_consensus.cc:697] T 328795a7f87644e0837e1d1519e1454d P 72001361a79346a7a6a5c755bdc62785 [term 2 LEADER]: Becoming Leader. State: Replica: 72001361a79346a7a6a5c755bdc62785, State: Running, Role: LEADER
I20260812 06:37:31.734934 13035 consensus_queue.cc:237] T 328795a7f87644e0837e1d1519e1454d P 72001361a79346a7a6a5c755bdc62785 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c9bb25173564fe383ecc4ee6d4be146" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 33169 } } peers { permanent_uuid: "72001361a79346a7a6a5c755bdc62785" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 45649 } } peers { permanent_uuid: "3b2b870051e548b7978192a6f3e375b9" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 39191 } }
I20260812 06:37:31.737901 11912 catalog_manager.cc:5719] T 328795a7f87644e0837e1d1519e1454d P 72001361a79346a7a6a5c755bdc62785 reported cstate change: term changed from 1 to 2, leader changed from 3b2b870051e548b7978192a6f3e375b9 (127.4.177.1) to 72001361a79346a7a6a5c755bdc62785 (127.4.177.2). New cstate: current_term: 2 leader_uuid: "72001361a79346a7a6a5c755bdc62785" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c9bb25173564fe383ecc4ee6d4be146" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 33169 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "72001361a79346a7a6a5c755bdc62785" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 45649 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3b2b870051e548b7978192a6f3e375b9" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 39191 } health_report { overall_health: UNKNOWN } } }
I20260812 06:37:31.744626 12169 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "328795a7f87644e0837e1d1519e1454d" candidate_uuid: "72001361a79346a7a6a5c755bdc62785" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "3b2b870051e548b7978192a6f3e375b9"
I20260812 06:37:31.744817 12169 raft_consensus.cc:1813] T 328795a7f87644e0837e1d1519e1454d P 3b2b870051e548b7978192a6f3e375b9 [term 1 LEADER]: Handling vote request from an unknown peer 72001361a79346a7a6a5c755bdc62785
I20260812 06:37:31.744903 12169 raft_consensus.cc:3055] T 328795a7f87644e0837e1d1519e1454d P 3b2b870051e548b7978192a6f3e375b9 [term 1 LEADER]: Stepping down as leader of term 1
I20260812 06:37:31.744959 12169 raft_consensus.cc:740] T 328795a7f87644e0837e1d1519e1454d P 3b2b870051e548b7978192a6f3e375b9 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 3b2b870051e548b7978192a6f3e375b9, State: Running, Role: LEADER
I20260812 06:37:31.745108 12169 consensus_queue.cc:260] T 328795a7f87644e0837e1d1519e1454d P 3b2b870051e548b7978192a6f3e375b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.2, Last appended by leader: 2, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "4c9bb25173564fe383ecc4ee6d4be146" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 33169 } } peers { permanent_uuid: "3b2b870051e548b7978192a6f3e375b9" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 39191 } }
I20260812 06:37:31.745285 12169 raft_consensus.cc:3060] T 328795a7f87644e0837e1d1519e1454d P 3b2b870051e548b7978192a6f3e375b9 [term 1 FOLLOWER]: Advancing to term 2
I20260812 06:37:31.746470 12169 raft_consensus.cc:2410] T 328795a7f87644e0837e1d1519e1454d P 3b2b870051e548b7978192a6f3e375b9 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 72001361a79346a7a6a5c755bdc62785 for term 2 because replica has last-logged OpId of term: 1 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260812 06:37:31.754678 12169 raft_consensus.cc:1275] T 328795a7f87644e0837e1d1519e1454d P 3b2b870051e548b7978192a6f3e375b9 [term 2 FOLLOWER]: Refusing update from remote peer 72001361a79346a7a6a5c755bdc62785: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260812 06:37:31.754758 13184 raft_consensus.cc:1275] T 328795a7f87644e0837e1d1519e1454d P 4c9bb25173564fe383ecc4ee6d4be146 [term 2 FOLLOWER]: Refusing update from remote peer 72001361a79346a7a6a5c755bdc62785: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260812 06:37:31.755270 13035 consensus_queue.cc:1048] T 328795a7f87644e0837e1d1519e1454d P 72001361a79346a7a6a5c755bdc62785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b2b870051e548b7978192a6f3e375b9" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 39191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260812 06:37:31.755414 13035 consensus_queue.cc:1243] T 328795a7f87644e0837e1d1519e1454d P 72001361a79346a7a6a5c755bdc62785 [LEADER]: Peer 3b2b870051e548b7978192a6f3e375b9 log is divergent from this leader: its last log entry 1.2 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1
I20260812 06:37:31.755492 13286 consensus_queue.cc:1048] T 328795a7f87644e0837e1d1519e1454d P 72001361a79346a7a6a5c755bdc62785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4c9bb25173564fe383ecc4ee6d4be146" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 33169 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260812 06:37:31.756539 12169 pending_rounds.cc:85] T 328795a7f87644e0837e1d1519e1454d P 3b2b870051e548b7978192a6f3e375b9: Aborting all ops after (but not including) 1
I20260812 06:37:31.756639 12169 pending_rounds.cc:107] T 328795a7f87644e0837e1d1519e1454d P 3b2b870051e548b7978192a6f3e375b9: Aborting uncommitted CHANGE_CONFIG_OP operation due to leader change: 1.2
I20260812 06:37:31.756706 12169 raft_consensus.cc:2917] T 328795a7f87644e0837e1d1519e1454d P 3b2b870051e548b7978192a6f3e375b9 [term 2 FOLLOWER]: Aborting config change with OpId 1.2: Aborted: Op aborted by new leader
I20260812 06:37:31.756791 12169 raft_consensus.cc:2889] T 328795a7f87644e0837e1d1519e1454d P 3b2b870051e548b7978192a6f3e375b9 [term 2 FOLLOWER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted by new leader
W20260812 06:37:31.756978 12169 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.ChangeConfig from 127.0.0.1:60666 (request call id 5) took 1406 ms (client timeout 1000 ms). Trace:
W20260812 06:37:31.757052 12169 rpcz_store.cc:269] 0812 06:37:30.350923 (+     0us) service_pool.cc:167] Inserting onto call queue
0812 06:37:30.351013 (+    90us) service_pool.cc:224] Handling call
0812 06:37:30.352344 (+  1331us) log.cc:844] Serialized 369 byte log entry
0812 06:37:31.756961 (+1404617us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":81,"raft.run_cpu_time_us":183,"raft.run_wall_time_us":185}
I20260812 06:37:31.765573 13308 mvcc.cc:204] Tried to move back new op lower bound from 7317572205580337152 to 7317572205509623808. Current Snapshot: MvccSnapshot[applied={T|T < 7317572205580337152}]
I20260812 06:37:31.816903 13151 tablet_service.cc:1467] Tablet server has 0 leaders and 0 scanners
I20260812 06:37:31.817492 12134 tablet_service.cc:1467] Tablet server has 0 leaders and 0 scanners
I20260812 06:37:31.817548 12910 tablet_service.cc:1467] Tablet server has 1 leaders and 0 scanners
Master Summary
               UUID               |      Address       | Status
----------------------------------+--------------------+---------
 159343eef8a04d51b49cdc4a4bedeef0 | 127.4.177.62:32865 | HEALTHY

Unusual flags for Master:
                         Flag                         |                                                                                          Value                                                                                           |      Tags       |         Master
------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------+-------------------------
 catalog_manager_wait_for_new_tablets_to_elect_leader | false                                                                                                                                                                                    | hidden          | all 1 server(s) checked
 ipki_ca_key_size                                     | 768                                                                                                                                                                                      | experimental    | all 1 server(s) checked
 ipki_server_key_size                                 | 768                                                                                                                                                                                      | experimental    | all 1 server(s) checked
 master_add_server_when_underreplicated               | false                                                                                                                                                                                    | hidden          | all 1 server(s) checked
 never_fsync                                          | true                                                                                                                                                                                     | unsafe,advanced | all 1 server(s) checked
 openssl_security_level_override                      | 0                                                                                                                                                                                        | unsafe,hidden   | all 1 server(s) checked
 rpc_reuseport                                        | true                                                                                                                                                                                     | experimental    | all 1 server(s) checked
 rpc_server_allow_ephemeral_ports                     | true                                                                                                                                                                                     | unsafe          | all 1 server(s) checked
 server_dump_info_format                              | pb                                                                                                                                                                                       | hidden          | all 1 server(s) checked
 server_dump_info_path                                | /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestReplaceChangeConfigOperation.1786516568546459-4804-0/raft_consensus-itest-cluster/master-0/data/info.pb | hidden          | all 1 server(s) checked
 tsk_num_rsa_bits                                     | 512                                                                                                                                                                                      | experimental    | all 1 server(s) checked

Flags of checked categories for Master:
        Flag         |       Value        |         Master
---------------------+--------------------+-------------------------
 builtin_ntp_servers | 127.4.177.20:41343 | all 1 server(s) checked
 time_source         | builtin            | all 1 server(s) checked

Tablet Server Summary
               UUID               |      Address      | Status  | Location | Tablet Leaders | Active Scanners
----------------------------------+-------------------+---------+----------+----------------+-----------------
 3b2b870051e548b7978192a6f3e375b9 | 127.4.177.1:39191 | HEALTHY | <none>   |       0        |       0
 4c9bb25173564fe383ecc4ee6d4be146 | 127.4.177.3:33169 | HEALTHY | <none>   |       0        |       0
 72001361a79346a7a6a5c755bdc62785 | 127.4.177.2:45649 | HEALTHY | <none>   |       1        |       0

Tablet Server Location Summary
 Location |  Count
----------+---------
 <none>   |       3

Unusual flags for Tablet Server:
               Flag               |                                                                                        Value                                                                                         |      Tags       |      Tablet Server
----------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------+-------------------------
 enable_leader_failure_detection  | false                                                                                                                                                                                | unsafe          | all 3 server(s) checked
 ipki_server_key_size             | 768                                                                                                                                                                                  | experimental    | all 3 server(s) checked
 local_ip_for_outbound_sockets    | 127.4.177.1                                                                                                                                                                          | experimental    | 127.4.177.1:39191
 local_ip_for_outbound_sockets    | 127.4.177.2                                                                                                                                                                          | experimental    | 127.4.177.2:45649
 local_ip_for_outbound_sockets    | 127.4.177.3                                                                                                                                                                          | experimental    | 127.4.177.3:33169
 never_fsync                      | true                                                                                                                                                                                 | unsafe,advanced | all 3 server(s) checked
 openssl_security_level_override  | 0                                                                                                                                                                                    | unsafe,hidden   | all 3 server(s) checked
 rpc_server_allow_ephemeral_ports | true                                                                                                                                                                                 | unsafe          | all 3 server(s) checked
 server_dump_info_format          | pb                                                                                                                                                                                   | hidden          | all 3 server(s) checked
 server_dump_info_path            | /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestReplaceChangeConfigOperation.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-0/data/info.pb | hidden          | 127.4.177.1:39191
 server_dump_info_path            | /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestReplaceChangeConfigOperation.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-1/data/info.pb | hidden          | 127.4.177.2:45649
 server_dump_info_path            | /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestReplaceChangeConfigOperation.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-2/data/info.pb | hidden          | 127.4.177.3:33169

Flags of checked categories for Tablet Server:
        Flag         |       Value        |      Tablet Server
---------------------+--------------------+-------------------------
 builtin_ntp_servers | 127.4.177.20:41343 | all 3 server(s) checked
 time_source         | builtin            | all 3 server(s) checked

Version Summary
     Version     |         Servers
-----------------+-------------------------
 1.19.0-SNAPSHOT | all 4 server(s) checked

Tablet Summary
The cluster doesn't have any matching system tables
Summary by table
   Name    | RF | Status  | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
-----------+----+---------+---------------+---------+------------+------------------+-------------
 TestTable | 3  | HEALTHY | 1             | 1       | 0          | 0                | 0

Tablet Replica Count Summary
   Statistic    | Replica Count
----------------+---------------
 Minimum        | 1
 First Quartile | 1
 Median         | 1
 Third Quartile | 1
 Maximum        | 1

Total Count Summary
                | Total Count
----------------+-------------
 Masters        | 1
 Tablet Servers | 3
 Tables         | 1
 Tablets        | 1
 Replicas       | 3

==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set

OK
I20260812 06:37:31.847333  4804 log_verifier.cc:126] Checking tablet 328795a7f87644e0837e1d1519e1454d
I20260812 06:37:31.870929  4804 log_verifier.cc:177] Verified matching terms for 3 ops in tablet 328795a7f87644e0837e1d1519e1454d
I20260812 06:37:32.286795 12939 consensus_queue.cc:237] T 328795a7f87644e0837e1d1519e1454d P 72001361a79346a7a6a5c755bdc62785 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "72001361a79346a7a6a5c755bdc62785" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 45649 } } peers { permanent_uuid: "3b2b870051e548b7978192a6f3e375b9" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 39191 } }
I20260812 06:37:32.288421 12169 raft_consensus.cc:1275] T 328795a7f87644e0837e1d1519e1454d P 3b2b870051e548b7978192a6f3e375b9 [term 2 FOLLOWER]: Refusing update from remote peer 72001361a79346a7a6a5c755bdc62785: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260812 06:37:32.288982 13364 consensus_queue.cc:1048] T 328795a7f87644e0837e1d1519e1454d P 72001361a79346a7a6a5c755bdc62785 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b2b870051e548b7978192a6f3e375b9" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 39191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260812 06:37:32.290967 13369 raft_consensus.cc:2955] T 328795a7f87644e0837e1d1519e1454d P 72001361a79346a7a6a5c755bdc62785 [term 2 LEADER]: Committing config change with OpId 2.4: config changed from index -1 to 4, VOTER 4c9bb25173564fe383ecc4ee6d4be146 (127.4.177.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "72001361a79346a7a6a5c755bdc62785" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 45649 } } peers { permanent_uuid: "3b2b870051e548b7978192a6f3e375b9" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 39191 } } }
I20260812 06:37:32.291445 12169 raft_consensus.cc:2955] T 328795a7f87644e0837e1d1519e1454d P 3b2b870051e548b7978192a6f3e375b9 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index -1 to 4, VOTER 4c9bb25173564fe383ecc4ee6d4be146 (127.4.177.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "72001361a79346a7a6a5c755bdc62785" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 45649 } } peers { permanent_uuid: "3b2b870051e548b7978192a6f3e375b9" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 39191 } } }
I20260812 06:37:32.294274 11913 catalog_manager.cc:5719] T 328795a7f87644e0837e1d1519e1454d P 72001361a79346a7a6a5c755bdc62785 reported cstate change: config changed from index -1 to 4, VOTER 4c9bb25173564fe383ecc4ee6d4be146 (127.4.177.3) evicted. New cstate: current_term: 2 leader_uuid: "72001361a79346a7a6a5c755bdc62785" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "72001361a79346a7a6a5c755bdc62785" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 45649 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3b2b870051e548b7978192a6f3e375b9" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 39191 } health_report { overall_health: HEALTHY } } }
I20260812 06:37:32.303797  4804 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskCnZu03/build/debug/bin/kudu with pid 12011
I20260812 06:37:32.303999 13151 tablet_service.cc:1558] Processing DeleteTablet for tablet 328795a7f87644e0837e1d1519e1454d with delete_type TABLET_DATA_TOMBSTONED (TS 4c9bb25173564fe383ecc4ee6d4be146 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:60202
I20260812 06:37:32.326558 13384 tablet_replica.cc:333] stopping tablet replica
I20260812 06:37:32.326936 13384 raft_consensus.cc:2243] T 328795a7f87644e0837e1d1519e1454d P 4c9bb25173564fe383ecc4ee6d4be146 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260812 06:37:32.327304 13384 raft_consensus.cc:2272] T 328795a7f87644e0837e1d1519e1454d P 4c9bb25173564fe383ecc4ee6d4be146 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260812 06:37:32.332790 13384 ts_tablet_manager.cc:1916] T 328795a7f87644e0837e1d1519e1454d P 4c9bb25173564fe383ecc4ee6d4be146: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260812 06:37:32.335988 13384 ts_tablet_manager.cc:1929] T 328795a7f87644e0837e1d1519e1454d P 4c9bb25173564fe383ecc4ee6d4be146: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.3
I20260812 06:37:32.336102 13384 log.cc:1199] T 328795a7f87644e0837e1d1519e1454d P 4c9bb25173564fe383ecc4ee6d4be146: Deleting WAL directory at /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestReplaceChangeConfigOperation.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-2/wal/wals/328795a7f87644e0837e1d1519e1454d
I20260812 06:37:32.336854 11893 catalog_manager.cc:5050] TS 4c9bb25173564fe383ecc4ee6d4be146 (127.4.177.3:33169): tablet 328795a7f87644e0837e1d1519e1454d (table TestTable [id=95751e2dde5243bba77e4c6a2a99358f]) successfully deleted
I20260812 06:37:32.338793  4804 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskCnZu03/build/debug/bin/kudu with pid 12802
I20260812 06:37:32.353571  4804 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskCnZu03/build/debug/bin/kudu with pid 13033
I20260812 06:37:32.370872  4804 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskCnZu03/build/debug/bin/kudu with pid 11857
2026-08-12T06:37:32Z chronyd exiting
[       OK ] RaftConsensusITest.TestReplaceChangeConfigOperation (3579 ms)
[ RUN      ] RaftConsensusITest.TestMemoryRemainsConstantDespiteTwoDeadFollowers
I20260812 06:37:32.407150  4804 ts_itest-base.cc:115] Starting cluster with:
I20260812 06:37:32.407227  4804 ts_itest-base.cc:116] --------------
I20260812 06:37:32.407245  4804 ts_itest-base.cc:117] 3 tablet servers
I20260812 06:37:32.407261  4804 ts_itest-base.cc:118] 3 replicas per TS
I20260812 06:37:32.407287  4804 ts_itest-base.cc:119] --------------
2026-08-12T06:37:32Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-08-12T06:37:32Z Disabled control of system clock
I20260812 06:37:32.417896  4804 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskCnZu03/build/debug/bin/kudu
/tmp/dist-test-taskCnZu03/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestMemoryRemainsConstantDespiteTwoDeadFollowers.1786516568546459-4804-0/raft_consensus-itest-cluster/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestMemoryRemainsConstantDespiteTwoDeadFollowers.1786516568546459-4804-0/raft_consensus-itest-cluster/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestMemoryRemainsConstantDespiteTwoDeadFollowers.1786516568546459-4804-0/raft_consensus-itest-cluster/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestMemoryRemainsConstantDespiteTwoDeadFollowers.1786516568546459-4804-0/raft_consensus-itest-cluster/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.177.62:42061
--webserver_interface=127.4.177.62
--webserver_port=0
--builtin_ntp_servers=127.4.177.20:36955
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.177.62:42061 with env {}
W20260812 06:37:32.525450 13404 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:37:32.525686 13404 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:37:32.525727 13404 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 06:37:32.529129 13404 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 06:37:32.529206 13404 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:37:32.529228 13404 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 06:37:32.529247 13404 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260812 06:37:32.533447 13404 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestMemoryRemainsConstantDespiteTwoDeadFollowers.1786516568546459-4804-0/raft_consensus-itest-cluster/master-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.177.20:36955
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestMemoryRemainsConstantDespiteTwoDeadFollowers.1786516568546459-4804-0/raft_consensus-itest-cluster/master-0/data
--fs_wal_dir=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestMemoryRemainsConstantDespiteTwoDeadFollowers.1786516568546459-4804-0/raft_consensus-itest-cluster/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.177.62:42061
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.177.62:42061
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestMemoryRemainsConstantDespiteTwoDeadFollowers.1786516568546459-4804-0/raft_consensus-itest-cluster/master-0/data/info.pb
--webserver_interface=127.4.177.62
--webserver_port=0
--never_fsync=true
--heap_profile_path=/tmp/kudu.13404
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false

Master server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 06:37:32.534624 13404 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:37:32.535792 13404 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 06:37:32.541640 13412 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:37:32.541710 13404 server_base.cc:1061] running on GCE node
W20260812 06:37:32.541798 13416 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:37:32.541724 13411 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:37:32.542560 13404 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:37:32.543574 13404 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:37:32.544763 13404 hybrid_clock.cc:648] HybridClock initialized: now 1786516652544751 us; error 36 us; skew 500 ppm
I20260812 06:37:32.546653 13404 webserver.cc:533] Webserver started at http://127.4.177.62:37657/ using document root <none> and password file <none>
I20260812 06:37:32.547241 13404 fs_manager.cc:362] Metadata directory not provided
I20260812 06:37:32.547327 13404 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:37:32.547564 13404 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:37:32.549172 13404 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestMemoryRemainsConstantDespiteTwoDeadFollowers.1786516568546459-4804-0/raft_consensus-itest-cluster/master-0/data/instance:
uuid: "1be9d28a076d424281ed5a5f76715d03"
format_stamp: "Formatted at 2026-08-12 06:37:32 on dist-test-slave-jztv"
I20260812 06:37:32.549686 13404 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestMemoryRemainsConstantDespiteTwoDeadFollowers.1786516568546459-4804-0/raft_consensus-itest-cluster/master-0/wal/instance:
uuid: "1be9d28a076d424281ed5a5f76715d03"
format_stamp: "Formatted at 2026-08-12 06:37:32 on dist-test-slave-jztv"
I20260812 06:37:32.553210 13404 fs_manager.cc:696] Time spent creating directory manager: real 0.003s	user 0.006s	sys 0.000s
I20260812 06:37:32.555557 13425 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 06:37:32.556704 13404 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.001s	sys 0.000s
I20260812 06:37:32.556849 13404 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestMemoryRemainsConstantDespiteTwoDeadFollowers.1786516568546459-4804-0/raft_consensus-itest-cluster/master-0/data,/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestMemoryRemainsConstantDespiteTwoDeadFollowers.1786516568546459-4804-0/raft_consensus-itest-cluster/master-0/wal
uuid: "1be9d28a076d424281ed5a5f76715d03"
format_stamp: "Formatted at 2026-08-12 06:37:32 on dist-test-slave-jztv"
I20260812 06:37:32.556960 13404 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestMemoryRemainsConstantDespiteTwoDeadFollowers.1786516568546459-4804-0/raft_consensus-itest-cluster/master-0/wal
metadata directory: /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestMemoryRemainsConstantDespiteTwoDeadFollowers.1786516568546459-4804-0/raft_consensus-itest-cluster/master-0/wal
1 data directories: /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestMemoryRemainsConstantDespiteTwoDeadFollowers.1786516568546459-4804-0/raft_consensus-itest-cluster/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:37:32.582633 13404 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 06:37:32.583395 13404 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:37:32.583612 13404 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:37:32.591311 13404 rpc_server.cc:307] RPC server started. Bound to: 127.4.177.62:42061
I20260812 06:37:32.591342 13516 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.4.177.62:42061 every 8 connection(s)
I20260812 06:37:32.592423 13404 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestMemoryRemainsConstantDespiteTwoDeadFollowers.1786516568546459-4804-0/raft_consensus-itest-cluster/master-0/data/info.pb
I20260812 06:37:32.594164  4804 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskCnZu03/build/debug/bin/kudu as pid 13404
I20260812 06:37:32.594316  4804 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestMemoryRemainsConstantDespiteTwoDeadFollowers.1786516568546459-4804-0/raft_consensus-itest-cluster/master-0/wal/instance
I20260812 06:37:32.595849 13519 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:37:32.600900 13519 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1be9d28a076d424281ed5a5f76715d03: Bootstrap starting.
I20260812 06:37:32.603152 13519 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1be9d28a076d424281ed5a5f76715d03: Neither blocks nor log segments found. Creating new log.
I20260812 06:37:32.603992 13519 log.cc:826] T 00000000000000000000000000000000 P 1be9d28a076d424281ed5a5f76715d03: Log is configured to *not* fsync() on all Append() calls
I20260812 06:37:32.605604 13519 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1be9d28a076d424281ed5a5f76715d03: No bootstrap required, opened a new log
I20260812 06:37:32.608454 13519 raft_consensus.cc:359] T 00000000000000000000000000000000 P 1be9d28a076d424281ed5a5f76715d03 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1be9d28a076d424281ed5a5f76715d03" member_type: VOTER last_known_addr { host: "127.4.177.62" port: 42061 } }
I20260812 06:37:32.608673 13519 raft_consensus.cc:385] T 00000000000000000000000000000000 P 1be9d28a076d424281ed5a5f76715d03 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:37:32.608757 13519 raft_consensus.cc:740] T 00000000000000000000000000000000 P 1be9d28a076d424281ed5a5f76715d03 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1be9d28a076d424281ed5a5f76715d03, State: Initialized, Role: FOLLOWER
I20260812 06:37:32.609156 13519 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1be9d28a076d424281ed5a5f76715d03 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1be9d28a076d424281ed5a5f76715d03" member_type: VOTER last_known_addr { host: "127.4.177.62" port: 42061 } }
I20260812 06:37:32.609300 13519 raft_consensus.cc:399] T 00000000000000000000000000000000 P 1be9d28a076d424281ed5a5f76715d03 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 06:37:32.609385 13519 raft_consensus.cc:493] T 00000000000000000000000000000000 P 1be9d28a076d424281ed5a5f76715d03 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 06:37:32.609517 13519 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 1be9d28a076d424281ed5a5f76715d03 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:37:32.610431 13519 raft_consensus.cc:515] T 00000000000000000000000000000000 P 1be9d28a076d424281ed5a5f76715d03 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1be9d28a076d424281ed5a5f76715d03" member_type: VOTER last_known_addr { host: "127.4.177.62" port: 42061 } }
I20260812 06:37:32.610770 13519 leader_election.cc:304] T 00000000000000000000000000000000 P 1be9d28a076d424281ed5a5f76715d03 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1be9d28a076d424281ed5a5f76715d03; no voters: 
I20260812 06:37:32.611070 13519 leader_election.cc:290] T 00000000000000000000000000000000 P 1be9d28a076d424281ed5a5f76715d03 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260812 06:37:32.611202 13527 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 1be9d28a076d424281ed5a5f76715d03 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 06:37:32.611442 13527 raft_consensus.cc:697] T 00000000000000000000000000000000 P 1be9d28a076d424281ed5a5f76715d03 [term 1 LEADER]: Becoming Leader. State: Replica: 1be9d28a076d424281ed5a5f76715d03, State: Running, Role: LEADER
I20260812 06:37:32.611764 13527 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1be9d28a076d424281ed5a5f76715d03 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1be9d28a076d424281ed5a5f76715d03" member_type: VOTER last_known_addr { host: "127.4.177.62" port: 42061 } }
I20260812 06:37:32.612288 13519 sys_catalog.cc:565] T 00000000000000000000000000000000 P 1be9d28a076d424281ed5a5f76715d03 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 06:37:32.613570 13529 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1be9d28a076d424281ed5a5f76715d03 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1be9d28a076d424281ed5a5f76715d03. Latest consensus state: current_term: 1 leader_uuid: "1be9d28a076d424281ed5a5f76715d03" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1be9d28a076d424281ed5a5f76715d03" member_type: VOTER last_known_addr { host: "127.4.177.62" port: 42061 } } }
I20260812 06:37:32.613556 13528 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1be9d28a076d424281ed5a5f76715d03 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1be9d28a076d424281ed5a5f76715d03" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1be9d28a076d424281ed5a5f76715d03" member_type: VOTER last_known_addr { host: "127.4.177.62" port: 42061 } } }
I20260812 06:37:32.613704 13529 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1be9d28a076d424281ed5a5f76715d03 [sys.catalog]: This master's current role is: LEADER
I20260812 06:37:32.613704 13528 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1be9d28a076d424281ed5a5f76715d03 [sys.catalog]: This master's current role is: LEADER
I20260812 06:37:32.614568 13538 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260812 06:37:32.617118 13538 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260812 06:37:32.622660 13538 catalog_manager.cc:1383] Generated new cluster ID: 73fd40ac69834ccab6af9e428af59378
I20260812 06:37:32.622741 13538 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260812 06:37:32.638634 13538 catalog_manager.cc:1406] Generated new certificate authority record
I20260812 06:37:32.639710 13538 catalog_manager.cc:1540] Loading token signing keys...
I20260812 06:37:32.647800 13538 catalog_manager.cc:6092] T 00000000000000000000000000000000 P 1be9d28a076d424281ed5a5f76715d03: Generated new TSK 0
I20260812 06:37:32.648558 13538 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260812 06:37:32.662802  4804 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskCnZu03/build/debug/bin/kudu
/tmp/dist-test-taskCnZu03/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestMemoryRemainsConstantDespiteTwoDeadFollowers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestMemoryRemainsConstantDespiteTwoDeadFollowers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestMemoryRemainsConstantDespiteTwoDeadFollowers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestMemoryRemainsConstantDespiteTwoDeadFollowers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.177.1:0
--local_ip_for_outbound_sockets=127.4.177.1
--webserver_interface=127.4.177.1
--webserver_port=0
--tserver_master_addrs=127.4.177.62:42061
--builtin_ntp_servers=127.4.177.20:36955
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--tablet_transaction_memory_limit_mb=2
--rpc_max_message_size=2097152 with env {}
W20260812 06:37:32.771128 13555 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:37:32.771356 13555 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:37:32.771445 13555 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 06:37:32.774741 13555 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:37:32.774899 13555 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.177.1
I20260812 06:37:32.778679 13555 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestMemoryRemainsConstantDespiteTwoDeadFollowers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.177.20:36955
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestMemoryRemainsConstantDespiteTwoDeadFollowers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestMemoryRemainsConstantDespiteTwoDeadFollowers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-0/wal
--rpc_max_message_size=2097152
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.177.1:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestMemoryRemainsConstantDespiteTwoDeadFollowers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--webserver_interface=127.4.177.1
--webserver_port=0
--tablet_transaction_memory_limit_mb=2
--tserver_master_addrs=127.4.177.62:42061
--never_fsync=true
--heap_profile_path=/tmp/kudu.13555
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.177.1

Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 06:37:32.779847 13555 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:37:32.780964 13555 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 06:37:32.787220 13563 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:37:32.787230 13567 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:37:32.787243 13565 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:37:32.787483 13555 server_base.cc:1061] running on GCE node
I20260812 06:37:32.787869 13555 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:37:32.788475 13555 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:37:32.789651 13555 hybrid_clock.cc:648] HybridClock initialized: now 1786516652789641 us; error 35 us; skew 500 ppm
I20260812 06:37:32.791508 13555 webserver.cc:533] Webserver started at http://127.4.177.1:40983/ using document root <none> and password file <none>
I20260812 06:37:32.792071 13555 fs_manager.cc:362] Metadata directory not provided
I20260812 06:37:32.792156 13555 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:37:32.792388 13555 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:37:32.793881 13555 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestMemoryRemainsConstantDespiteTwoDeadFollowers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-0/data/instance:
uuid: "dac5c1c596ae47d99b88e0016c6bcfb9"
format_stamp: "Formatted at 2026-08-12 06:37:32 on dist-test-slave-jztv"
I20260812 06:37:32.794431 13555 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestMemoryRemainsConstantDespiteTwoDeadFollowers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-0/wal/instance:
uuid: "dac5c1c596ae47d99b88e0016c6bcfb9"
format_stamp: "Formatted at 2026-08-12 06:37:32 on dist-test-slave-jztv"
I20260812 06:37:32.797739 13555 fs_manager.cc:696] Time spent creating directory manager: real 0.003s	user 0.002s	sys 0.001s
I20260812 06:37:32.800086 13578 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 06:37:32.801075 13555 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.001s	sys 0.000s
I20260812 06:37:32.801206 13555 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestMemoryRemainsConstantDespiteTwoDeadFollowers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-0/data,/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestMemoryRemainsConstantDespiteTwoDeadFollowers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-0/wal
uuid: "dac5c1c596ae47d99b88e0016c6bcfb9"
format_stamp: "Formatted at 2026-08-12 06:37:32 on dist-test-slave-jztv"
I20260812 06:37:32.801311 13555 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestMemoryRemainsConstantDespiteTwoDeadFollowers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-0/wal
metadata directory: /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestMemoryRemainsConstantDespiteTwoDeadFollowers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-0/wal
1 data directories: /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestMemoryRemainsConstantDespiteTwoDeadFollowers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:37:32.814966 13555 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 06:37:32.815587 13555 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:37:32.815781 13555 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:37:32.816341 13555 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260812 06:37:32.817332 13555 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:37:32.817404 13555 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260812 06:37:32.817467 13555 ts_tablet_manager.cc:616] Registered 0 tablets
I20260812 06:37:32.817510 13555 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260812 06:37:32.826470 13555 rpc_server.cc:307] RPC server started. Bound to: 127.4.177.1:45819
I20260812 06:37:32.826524 13783 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.4.177.1:45819 every 8 connection(s)
I20260812 06:37:32.827375 13555 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestMemoryRemainsConstantDespiteTwoDeadFollowers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-0/data/info.pb
I20260812 06:37:32.830355  4804 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskCnZu03/build/debug/bin/kudu as pid 13555
I20260812 06:37:32.830484  4804 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestMemoryRemainsConstantDespiteTwoDeadFollowers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-0/wal/instance
I20260812 06:37:32.832715  4804 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskCnZu03/build/debug/bin/kudu
/tmp/dist-test-taskCnZu03/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestMemoryRemainsConstantDespiteTwoDeadFollowers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestMemoryRemainsConstantDespiteTwoDeadFollowers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestMemoryRemainsConstantDespiteTwoDeadFollowers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestMemoryRemainsConstantDespiteTwoDeadFollowers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.177.2:0
--local_ip_for_outbound_sockets=127.4.177.2
--webserver_interface=127.4.177.2
--webserver_port=0
--tserver_master_addrs=127.4.177.62:42061
--builtin_ntp_servers=127.4.177.20:36955
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--tablet_transaction_memory_limit_mb=2
--rpc_max_message_size=2097152 with env {}
I20260812 06:37:32.836004 13784 heartbeater.cc:344] Connected to a master server at 127.4.177.62:42061
I20260812 06:37:32.836220 13784 heartbeater.cc:461] Registering TS with master...
I20260812 06:37:32.836699 13784 heartbeater.cc:507] Master 127.4.177.62:42061 requested a full tablet report, sending...
I20260812 06:37:32.838090 13453 ts_manager.cc:194] Registered new tserver with Master: dac5c1c596ae47d99b88e0016c6bcfb9 (127.4.177.1:45819)
I20260812 06:37:32.839659 13453 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.4.177.1:44617
W20260812 06:37:32.945007 13788 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:37:32.945271 13788 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:37:32.945351 13788 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 06:37:32.948680 13788 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:37:32.948843 13788 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.177.2
I20260812 06:37:32.952540 13788 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestMemoryRemainsConstantDespiteTwoDeadFollowers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-1/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.177.20:36955
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestMemoryRemainsConstantDespiteTwoDeadFollowers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestMemoryRemainsConstantDespiteTwoDeadFollowers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-1/wal
--rpc_max_message_size=2097152
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.177.2:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestMemoryRemainsConstantDespiteTwoDeadFollowers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--webserver_interface=127.4.177.2
--webserver_port=0
--tablet_transaction_memory_limit_mb=2
--tserver_master_addrs=127.4.177.62:42061
--never_fsync=true
--heap_profile_path=/tmp/kudu.13788
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.177.2

Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 06:37:32.953635 13788 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:37:32.954823 13788 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 06:37:32.961292 13798 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:37:32.961339 13796 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:37:32.961382 13800 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:37:32.961504 13788 server_base.cc:1061] running on GCE node
I20260812 06:37:32.961859 13788 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:37:32.962476 13788 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:37:32.963675 13788 hybrid_clock.cc:648] HybridClock initialized: now 1786516652963662 us; error 33 us; skew 500 ppm
I20260812 06:37:32.965370 13788 webserver.cc:533] Webserver started at http://127.4.177.2:40459/ using document root <none> and password file <none>
I20260812 06:37:32.965939 13788 fs_manager.cc:362] Metadata directory not provided
I20260812 06:37:32.966027 13788 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:37:32.966284 13788 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:37:32.967819 13788 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestMemoryRemainsConstantDespiteTwoDeadFollowers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-1/data/instance:
uuid: "0d424b3961bf4d859028eed7f3748eff"
format_stamp: "Formatted at 2026-08-12 06:37:32 on dist-test-slave-jztv"
I20260812 06:37:32.968307 13788 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestMemoryRemainsConstantDespiteTwoDeadFollowers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-1/wal/instance:
uuid: "0d424b3961bf4d859028eed7f3748eff"
format_stamp: "Formatted at 2026-08-12 06:37:32 on dist-test-slave-jztv"
I20260812 06:37:32.971714 13788 fs_manager.cc:696] Time spent creating directory manager: real 0.003s	user 0.003s	sys 0.002s
I20260812 06:37:32.973822 13814 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 06:37:32.974905 13788 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.001s
I20260812 06:37:32.975044 13788 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestMemoryRemainsConstantDespiteTwoDeadFollowers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-1/data,/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestMemoryRemainsConstantDespiteTwoDeadFollowers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-1/wal
uuid: "0d424b3961bf4d859028eed7f3748eff"
format_stamp: "Formatted at 2026-08-12 06:37:32 on dist-test-slave-jztv"
I20260812 06:37:32.975150 13788 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestMemoryRemainsConstantDespiteTwoDeadFollowers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-1/wal
metadata directory: /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestMemoryRemainsConstantDespiteTwoDeadFollowers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-1/wal
1 data directories: /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestMemoryRemainsConstantDespiteTwoDeadFollowers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:37:32.991801 13788 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 06:37:32.992517 13788 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:37:32.992729 13788 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:37:32.993314 13788 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260812 06:37:32.994415 13788 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:37:32.994488 13788 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260812 06:37:32.994585 13788 ts_tablet_manager.cc:616] Registered 0 tablets
I20260812 06:37:32.994630 13788 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260812 06:37:33.004731 13788 rpc_server.cc:307] RPC server started. Bound to: 127.4.177.2:37655
I20260812 06:37:33.004788 14006 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.4.177.2:37655 every 8 connection(s)
I20260812 06:37:33.005815 13788 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestMemoryRemainsConstantDespiteTwoDeadFollowers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-1/data/info.pb
I20260812 06:37:33.010649  4804 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskCnZu03/build/debug/bin/kudu as pid 13788
I20260812 06:37:33.010815  4804 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestMemoryRemainsConstantDespiteTwoDeadFollowers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-1/wal/instance
I20260812 06:37:33.013216  4804 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskCnZu03/build/debug/bin/kudu
/tmp/dist-test-taskCnZu03/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestMemoryRemainsConstantDespiteTwoDeadFollowers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestMemoryRemainsConstantDespiteTwoDeadFollowers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestMemoryRemainsConstantDespiteTwoDeadFollowers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestMemoryRemainsConstantDespiteTwoDeadFollowers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.177.3:0
--local_ip_for_outbound_sockets=127.4.177.3
--webserver_interface=127.4.177.3
--webserver_port=0
--tserver_master_addrs=127.4.177.62:42061
--builtin_ntp_servers=127.4.177.20:36955
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--tablet_transaction_memory_limit_mb=2
--rpc_max_message_size=2097152 with env {}
I20260812 06:37:33.014652 14010 heartbeater.cc:344] Connected to a master server at 127.4.177.62:42061
I20260812 06:37:33.014914 14010 heartbeater.cc:461] Registering TS with master...
I20260812 06:37:33.015429 14010 heartbeater.cc:507] Master 127.4.177.62:42061 requested a full tablet report, sending...
I20260812 06:37:33.016491 13453 ts_manager.cc:194] Registered new tserver with Master: 0d424b3961bf4d859028eed7f3748eff (127.4.177.2:37655)
I20260812 06:37:33.017078 13453 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.4.177.2:47721
W20260812 06:37:33.119755 14016 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:37:33.120035 14016 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:37:33.120136 14016 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 06:37:33.123594 14016 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:37:33.123761 14016 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.177.3
I20260812 06:37:33.127796 14016 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestMemoryRemainsConstantDespiteTwoDeadFollowers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-2/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.177.20:36955
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestMemoryRemainsConstantDespiteTwoDeadFollowers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestMemoryRemainsConstantDespiteTwoDeadFollowers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-2/wal
--rpc_max_message_size=2097152
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.177.3:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestMemoryRemainsConstantDespiteTwoDeadFollowers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--webserver_interface=127.4.177.3
--webserver_port=0
--tablet_transaction_memory_limit_mb=2
--tserver_master_addrs=127.4.177.62:42061
--never_fsync=true
--heap_profile_path=/tmp/kudu.14016
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.177.3

Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 06:37:33.129007 14016 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:37:33.130158 14016 file_cache.cc:504] Constructed file cache file cache with capacity 419430
I20260812 06:37:33.137147 14016 server_base.cc:1061] running on GCE node
W20260812 06:37:33.137099 14028 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:37:33.137101 14023 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:37:33.137086 14030 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:37:33.137737 14016 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:37:33.138372 14016 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:37:33.139552 14016 hybrid_clock.cc:648] HybridClock initialized: now 1786516653139536 us; error 33 us; skew 500 ppm
I20260812 06:37:33.141485 14016 webserver.cc:533] Webserver started at http://127.4.177.3:36563/ using document root <none> and password file <none>
I20260812 06:37:33.142117 14016 fs_manager.cc:362] Metadata directory not provided
I20260812 06:37:33.142177 14016 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:37:33.142422 14016 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:37:33.144025 14016 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestMemoryRemainsConstantDespiteTwoDeadFollowers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-2/data/instance:
uuid: "fd09b24d2a674c80968d508eb9636439"
format_stamp: "Formatted at 2026-08-12 06:37:33 on dist-test-slave-jztv"
I20260812 06:37:33.144604 14016 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestMemoryRemainsConstantDespiteTwoDeadFollowers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-2/wal/instance:
uuid: "fd09b24d2a674c80968d508eb9636439"
format_stamp: "Formatted at 2026-08-12 06:37:33 on dist-test-slave-jztv"
I20260812 06:37:33.148221 14016 fs_manager.cc:696] Time spent creating directory manager: real 0.003s	user 0.005s	sys 0.000s
I20260812 06:37:33.150666 14041 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 06:37:33.151750 14016 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260812 06:37:33.151873 14016 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestMemoryRemainsConstantDespiteTwoDeadFollowers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-2/data,/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestMemoryRemainsConstantDespiteTwoDeadFollowers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-2/wal
uuid: "fd09b24d2a674c80968d508eb9636439"
format_stamp: "Formatted at 2026-08-12 06:37:33 on dist-test-slave-jztv"
I20260812 06:37:33.151984 14016 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestMemoryRemainsConstantDespiteTwoDeadFollowers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-2/wal
metadata directory: /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestMemoryRemainsConstantDespiteTwoDeadFollowers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-2/wal
1 data directories: /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestMemoryRemainsConstantDespiteTwoDeadFollowers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:37:33.181203 14016 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 06:37:33.181983 14016 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:37:33.182200 14016 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:37:33.182864 14016 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260812 06:37:33.183907 14016 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:37:33.183961 14016 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260812 06:37:33.184033 14016 ts_tablet_manager.cc:616] Registered 0 tablets
I20260812 06:37:33.184073 14016 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260812 06:37:33.194187 14016 rpc_server.cc:307] RPC server started. Bound to: 127.4.177.3:43793
I20260812 06:37:33.194231 14244 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.4.177.3:43793 every 8 connection(s)
I20260812 06:37:33.195209 14016 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestMemoryRemainsConstantDespiteTwoDeadFollowers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-2/data/info.pb
I20260812 06:37:33.201172  4804 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskCnZu03/build/debug/bin/kudu as pid 14016
I20260812 06:37:33.201329  4804 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestMemoryRemainsConstantDespiteTwoDeadFollowers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-2/wal/instance
I20260812 06:37:33.204106 14246 heartbeater.cc:344] Connected to a master server at 127.4.177.62:42061
I20260812 06:37:33.204401 14246 heartbeater.cc:461] Registering TS with master...
I20260812 06:37:33.204962 14246 heartbeater.cc:507] Master 127.4.177.62:42061 requested a full tablet report, sending...
I20260812 06:37:33.206246 13453 ts_manager.cc:194] Registered new tserver with Master: fd09b24d2a674c80968d508eb9636439 (127.4.177.3:43793)
I20260812 06:37:33.206888 13453 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.4.177.3:46595
I20260812 06:37:33.216185  4804 external_mini_cluster.cc:955] 3 TS(s) registered with all masters
I20260812 06:37:33.225996 13453 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:41372:
name: "TestTable"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
owner: "alice"
W20260812 06:37:33.228615 13453 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table TestTable in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 06:37:33.246408 13898 tablet_service.cc:1511] Processing CreateTablet for tablet fcde898f35224c75ae069a036da1f9df (DEFAULT_TABLE table=TestTable [id=071c2cbfc1344d18883fc45f80604601]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 06:37:33.246408 14126 tablet_service.cc:1511] Processing CreateTablet for tablet fcde898f35224c75ae069a036da1f9df (DEFAULT_TABLE table=TestTable [id=071c2cbfc1344d18883fc45f80604601]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 06:37:33.246767 13667 tablet_service.cc:1511] Processing CreateTablet for tablet fcde898f35224c75ae069a036da1f9df (DEFAULT_TABLE table=TestTable [id=071c2cbfc1344d18883fc45f80604601]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 06:37:33.247702 13667 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fcde898f35224c75ae069a036da1f9df. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:37:33.247874 13898 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fcde898f35224c75ae069a036da1f9df. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:37:33.247874 14126 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fcde898f35224c75ae069a036da1f9df. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:37:33.253082 14275 tablet_bootstrap.cc:492] T fcde898f35224c75ae069a036da1f9df P dac5c1c596ae47d99b88e0016c6bcfb9: Bootstrap starting.
I20260812 06:37:33.254158 14279 tablet_bootstrap.cc:492] T fcde898f35224c75ae069a036da1f9df P fd09b24d2a674c80968d508eb9636439: Bootstrap starting.
I20260812 06:37:33.255735 14275 tablet_bootstrap.cc:654] T fcde898f35224c75ae069a036da1f9df P dac5c1c596ae47d99b88e0016c6bcfb9: Neither blocks nor log segments found. Creating new log.
I20260812 06:37:33.256043 14279 tablet_bootstrap.cc:654] T fcde898f35224c75ae069a036da1f9df P fd09b24d2a674c80968d508eb9636439: Neither blocks nor log segments found. Creating new log.
I20260812 06:37:33.256567 14275 log.cc:826] T fcde898f35224c75ae069a036da1f9df P dac5c1c596ae47d99b88e0016c6bcfb9: Log is configured to *not* fsync() on all Append() calls
I20260812 06:37:33.256779 14279 log.cc:826] T fcde898f35224c75ae069a036da1f9df P fd09b24d2a674c80968d508eb9636439: Log is configured to *not* fsync() on all Append() calls
I20260812 06:37:33.257452 14283 tablet_bootstrap.cc:492] T fcde898f35224c75ae069a036da1f9df P 0d424b3961bf4d859028eed7f3748eff: Bootstrap starting.
I20260812 06:37:33.258579 14275 tablet_bootstrap.cc:492] T fcde898f35224c75ae069a036da1f9df P dac5c1c596ae47d99b88e0016c6bcfb9: No bootstrap required, opened a new log
I20260812 06:37:33.258579 14279 tablet_bootstrap.cc:492] T fcde898f35224c75ae069a036da1f9df P fd09b24d2a674c80968d508eb9636439: No bootstrap required, opened a new log
I20260812 06:37:33.258783 14279 ts_tablet_manager.cc:1403] T fcde898f35224c75ae069a036da1f9df P fd09b24d2a674c80968d508eb9636439: Time spent bootstrapping tablet: real 0.005s	user 0.001s	sys 0.003s
I20260812 06:37:33.258783 14275 ts_tablet_manager.cc:1403] T fcde898f35224c75ae069a036da1f9df P dac5c1c596ae47d99b88e0016c6bcfb9: Time spent bootstrapping tablet: real 0.006s	user 0.001s	sys 0.004s
I20260812 06:37:33.259563 14283 tablet_bootstrap.cc:654] T fcde898f35224c75ae069a036da1f9df P 0d424b3961bf4d859028eed7f3748eff: Neither blocks nor log segments found. Creating new log.
I20260812 06:37:33.260377 14283 log.cc:826] T fcde898f35224c75ae069a036da1f9df P 0d424b3961bf4d859028eed7f3748eff: Log is configured to *not* fsync() on all Append() calls
I20260812 06:37:33.262084 14283 tablet_bootstrap.cc:492] T fcde898f35224c75ae069a036da1f9df P 0d424b3961bf4d859028eed7f3748eff: No bootstrap required, opened a new log
I20260812 06:37:33.262012 14279 raft_consensus.cc:359] T fcde898f35224c75ae069a036da1f9df P fd09b24d2a674c80968d508eb9636439 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dac5c1c596ae47d99b88e0016c6bcfb9" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 45819 } } peers { permanent_uuid: "0d424b3961bf4d859028eed7f3748eff" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 37655 } } peers { permanent_uuid: "fd09b24d2a674c80968d508eb9636439" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 43793 } }
I20260812 06:37:33.262013 14275 raft_consensus.cc:359] T fcde898f35224c75ae069a036da1f9df P dac5c1c596ae47d99b88e0016c6bcfb9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dac5c1c596ae47d99b88e0016c6bcfb9" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 45819 } } peers { permanent_uuid: "0d424b3961bf4d859028eed7f3748eff" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 37655 } } peers { permanent_uuid: "fd09b24d2a674c80968d508eb9636439" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 43793 } }
I20260812 06:37:33.262262 14279 raft_consensus.cc:385] T fcde898f35224c75ae069a036da1f9df P fd09b24d2a674c80968d508eb9636439 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:37:33.262262 14275 raft_consensus.cc:385] T fcde898f35224c75ae069a036da1f9df P dac5c1c596ae47d99b88e0016c6bcfb9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:37:33.262331 14279 raft_consensus.cc:740] T fcde898f35224c75ae069a036da1f9df P fd09b24d2a674c80968d508eb9636439 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fd09b24d2a674c80968d508eb9636439, State: Initialized, Role: FOLLOWER
I20260812 06:37:33.262342 14275 raft_consensus.cc:740] T fcde898f35224c75ae069a036da1f9df P dac5c1c596ae47d99b88e0016c6bcfb9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dac5c1c596ae47d99b88e0016c6bcfb9, State: Initialized, Role: FOLLOWER
I20260812 06:37:33.262351 14283 ts_tablet_manager.cc:1403] T fcde898f35224c75ae069a036da1f9df P 0d424b3961bf4d859028eed7f3748eff: Time spent bootstrapping tablet: real 0.005s	user 0.001s	sys 0.004s
I20260812 06:37:33.262821 14279 consensus_queue.cc:260] T fcde898f35224c75ae069a036da1f9df P fd09b24d2a674c80968d508eb9636439 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dac5c1c596ae47d99b88e0016c6bcfb9" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 45819 } } peers { permanent_uuid: "0d424b3961bf4d859028eed7f3748eff" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 37655 } } peers { permanent_uuid: "fd09b24d2a674c80968d508eb9636439" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 43793 } }
I20260812 06:37:33.262821 14275 consensus_queue.cc:260] T fcde898f35224c75ae069a036da1f9df P dac5c1c596ae47d99b88e0016c6bcfb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dac5c1c596ae47d99b88e0016c6bcfb9" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 45819 } } peers { permanent_uuid: "0d424b3961bf4d859028eed7f3748eff" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 37655 } } peers { permanent_uuid: "fd09b24d2a674c80968d508eb9636439" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 43793 } }
I20260812 06:37:33.263540 13784 heartbeater.cc:499] Master 127.4.177.62:42061 was elected leader, sending a full tablet report...
I20260812 06:37:33.263757 14279 ts_tablet_manager.cc:1434] T fcde898f35224c75ae069a036da1f9df P fd09b24d2a674c80968d508eb9636439: Time spent starting tablet: real 0.005s	user 0.002s	sys 0.003s
I20260812 06:37:33.263842 14246 heartbeater.cc:499] Master 127.4.177.62:42061 was elected leader, sending a full tablet report...
I20260812 06:37:33.264781 14275 ts_tablet_manager.cc:1434] T fcde898f35224c75ae069a036da1f9df P dac5c1c596ae47d99b88e0016c6bcfb9: Time spent starting tablet: real 0.006s	user 0.002s	sys 0.003s
I20260812 06:37:33.265540 14283 raft_consensus.cc:359] T fcde898f35224c75ae069a036da1f9df P 0d424b3961bf4d859028eed7f3748eff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dac5c1c596ae47d99b88e0016c6bcfb9" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 45819 } } peers { permanent_uuid: "0d424b3961bf4d859028eed7f3748eff" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 37655 } } peers { permanent_uuid: "fd09b24d2a674c80968d508eb9636439" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 43793 } }
I20260812 06:37:33.265826 14283 raft_consensus.cc:385] T fcde898f35224c75ae069a036da1f9df P 0d424b3961bf4d859028eed7f3748eff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:37:33.265899 14283 raft_consensus.cc:740] T fcde898f35224c75ae069a036da1f9df P 0d424b3961bf4d859028eed7f3748eff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d424b3961bf4d859028eed7f3748eff, State: Initialized, Role: FOLLOWER
I20260812 06:37:33.266376 14283 consensus_queue.cc:260] T fcde898f35224c75ae069a036da1f9df P 0d424b3961bf4d859028eed7f3748eff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dac5c1c596ae47d99b88e0016c6bcfb9" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 45819 } } peers { permanent_uuid: "0d424b3961bf4d859028eed7f3748eff" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 37655 } } peers { permanent_uuid: "fd09b24d2a674c80968d508eb9636439" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 43793 } }
I20260812 06:37:33.267251 14283 ts_tablet_manager.cc:1434] T fcde898f35224c75ae069a036da1f9df P 0d424b3961bf4d859028eed7f3748eff: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260812 06:37:33.267583 14010 heartbeater.cc:499] Master 127.4.177.62:42061 was elected leader, sending a full tablet report...
I20260812 06:37:33.279047 14290 raft_consensus.cc:493] T fcde898f35224c75ae069a036da1f9df P fd09b24d2a674c80968d508eb9636439 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 06:37:33.279250 14290 raft_consensus.cc:515] T fcde898f35224c75ae069a036da1f9df P fd09b24d2a674c80968d508eb9636439 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dac5c1c596ae47d99b88e0016c6bcfb9" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 45819 } } peers { permanent_uuid: "0d424b3961bf4d859028eed7f3748eff" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 37655 } } peers { permanent_uuid: "fd09b24d2a674c80968d508eb9636439" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 43793 } }
I20260812 06:37:33.280421 14290 leader_election.cc:290] T fcde898f35224c75ae069a036da1f9df P fd09b24d2a674c80968d508eb9636439 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers dac5c1c596ae47d99b88e0016c6bcfb9 (127.4.177.1:45819), 0d424b3961bf4d859028eed7f3748eff (127.4.177.2:37655)
I20260812 06:37:33.285079 13705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fcde898f35224c75ae069a036da1f9df" candidate_uuid: "fd09b24d2a674c80968d508eb9636439" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dac5c1c596ae47d99b88e0016c6bcfb9" is_pre_election: true
I20260812 06:37:33.285372 13705 raft_consensus.cc:2468] T fcde898f35224c75ae069a036da1f9df P dac5c1c596ae47d99b88e0016c6bcfb9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fd09b24d2a674c80968d508eb9636439 in term 0.
I20260812 06:37:33.285916 14045 leader_election.cc:304] T fcde898f35224c75ae069a036da1f9df P fd09b24d2a674c80968d508eb9636439 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: dac5c1c596ae47d99b88e0016c6bcfb9, fd09b24d2a674c80968d508eb9636439; no voters: 
I20260812 06:37:33.286103 13932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fcde898f35224c75ae069a036da1f9df" candidate_uuid: "fd09b24d2a674c80968d508eb9636439" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d424b3961bf4d859028eed7f3748eff" is_pre_election: true
I20260812 06:37:33.286166 14290 raft_consensus.cc:2804] T fcde898f35224c75ae069a036da1f9df P fd09b24d2a674c80968d508eb9636439 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 06:37:33.286255 14290 raft_consensus.cc:493] T fcde898f35224c75ae069a036da1f9df P fd09b24d2a674c80968d508eb9636439 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 06:37:33.286321 14290 raft_consensus.cc:3060] T fcde898f35224c75ae069a036da1f9df P fd09b24d2a674c80968d508eb9636439 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:37:33.286391 13932 raft_consensus.cc:2468] T fcde898f35224c75ae069a036da1f9df P 0d424b3961bf4d859028eed7f3748eff [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fd09b24d2a674c80968d508eb9636439 in term 0.
I20260812 06:37:33.287415 14290 raft_consensus.cc:515] T fcde898f35224c75ae069a036da1f9df P fd09b24d2a674c80968d508eb9636439 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dac5c1c596ae47d99b88e0016c6bcfb9" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 45819 } } peers { permanent_uuid: "0d424b3961bf4d859028eed7f3748eff" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 37655 } } peers { permanent_uuid: "fd09b24d2a674c80968d508eb9636439" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 43793 } }
I20260812 06:37:33.287760 14290 leader_election.cc:290] T fcde898f35224c75ae069a036da1f9df P fd09b24d2a674c80968d508eb9636439 [CANDIDATE]: Term 1 election: Requested vote from peers dac5c1c596ae47d99b88e0016c6bcfb9 (127.4.177.1:45819), 0d424b3961bf4d859028eed7f3748eff (127.4.177.2:37655)
I20260812 06:37:33.288102 13705 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fcde898f35224c75ae069a036da1f9df" candidate_uuid: "fd09b24d2a674c80968d508eb9636439" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dac5c1c596ae47d99b88e0016c6bcfb9"
I20260812 06:37:33.288223 13705 raft_consensus.cc:3060] T fcde898f35224c75ae069a036da1f9df P dac5c1c596ae47d99b88e0016c6bcfb9 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:37:33.288188 13932 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fcde898f35224c75ae069a036da1f9df" candidate_uuid: "fd09b24d2a674c80968d508eb9636439" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d424b3961bf4d859028eed7f3748eff"
I20260812 06:37:33.288347 13932 raft_consensus.cc:3060] T fcde898f35224c75ae069a036da1f9df P 0d424b3961bf4d859028eed7f3748eff [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:37:33.289439 13705 raft_consensus.cc:2468] T fcde898f35224c75ae069a036da1f9df P dac5c1c596ae47d99b88e0016c6bcfb9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fd09b24d2a674c80968d508eb9636439 in term 1.
I20260812 06:37:33.289616 13932 raft_consensus.cc:2468] T fcde898f35224c75ae069a036da1f9df P 0d424b3961bf4d859028eed7f3748eff [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fd09b24d2a674c80968d508eb9636439 in term 1.
I20260812 06:37:33.289789 14045 leader_election.cc:304] T fcde898f35224c75ae069a036da1f9df P fd09b24d2a674c80968d508eb9636439 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: dac5c1c596ae47d99b88e0016c6bcfb9, fd09b24d2a674c80968d508eb9636439; no voters: 
I20260812 06:37:33.289989 14290 raft_consensus.cc:2804] T fcde898f35224c75ae069a036da1f9df P fd09b24d2a674c80968d508eb9636439 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 06:37:33.290285 14290 raft_consensus.cc:697] T fcde898f35224c75ae069a036da1f9df P fd09b24d2a674c80968d508eb9636439 [term 1 LEADER]: Becoming Leader. State: Replica: fd09b24d2a674c80968d508eb9636439, State: Running, Role: LEADER
I20260812 06:37:33.290596 14290 consensus_queue.cc:237] T fcde898f35224c75ae069a036da1f9df P fd09b24d2a674c80968d508eb9636439 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dac5c1c596ae47d99b88e0016c6bcfb9" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 45819 } } peers { permanent_uuid: "0d424b3961bf4d859028eed7f3748eff" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 37655 } } peers { permanent_uuid: "fd09b24d2a674c80968d508eb9636439" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 43793 } }
I20260812 06:37:33.293623 13451 catalog_manager.cc:5719] T fcde898f35224c75ae069a036da1f9df P fd09b24d2a674c80968d508eb9636439 reported cstate change: term changed from 0 to 1, leader changed from <none> to fd09b24d2a674c80968d508eb9636439 (127.4.177.3). New cstate: current_term: 1 leader_uuid: "fd09b24d2a674c80968d508eb9636439" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dac5c1c596ae47d99b88e0016c6bcfb9" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 45819 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0d424b3961bf4d859028eed7f3748eff" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 37655 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fd09b24d2a674c80968d508eb9636439" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 43793 } health_report { overall_health: HEALTHY } } }
I20260812 06:37:33.302871  4804 external_mini_cluster.cc:955] 3 TS(s) registered with all masters
I20260812 06:37:33.304391  4804 ts_itest-base.cc:246] Waiting for 1 tablets on tserver dac5c1c596ae47d99b88e0016c6bcfb9 to finish bootstrapping
I20260812 06:37:33.310195  4804 ts_itest-base.cc:246] Waiting for 1 tablets on tserver 0d424b3961bf4d859028eed7f3748eff to finish bootstrapping
I20260812 06:37:33.316017  4804 ts_itest-base.cc:246] Waiting for 1 tablets on tserver fd09b24d2a674c80968d508eb9636439 to finish bootstrapping
I20260812 06:37:33.327237  4804 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskCnZu03/build/debug/bin/kudu with pid 13555
I20260812 06:37:33.344131  4804 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskCnZu03/build/debug/bin/kudu with pid 13788
I20260812 06:37:33.373464  4804 test_workload.cc:405] TestWorkload: Skipping table creation because table TestTable already exists
W20260812 06:37:33.411988 14083 op_tracker.cc:165] op on tablet fcde898f35224c75ae069a036da1f9df rejected due to memory pressure: the memory usage of this op (1502305) plus the current consumption (1502305) exceeds the op memory limit (2097152) or the limit of an ancestral memory tracker.
W20260812 06:37:33.416332 14045 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.4.177.1:45819: connect: Connection refused (error 111)
W20260812 06:37:33.417382 14045 consensus_peers.cc:597] T fcde898f35224c75ae069a036da1f9df P fd09b24d2a674c80968d508eb9636439 -> Peer dac5c1c596ae47d99b88e0016c6bcfb9 (127.4.177.1:45819): Couldn't send request to peer dac5c1c596ae47d99b88e0016c6bcfb9. Status: Network error: Client connection negotiation failed: client connection to 127.4.177.1:45819: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260812 06:37:33.417528 14045 consensus_peers.cc:597] T fcde898f35224c75ae069a036da1f9df P fd09b24d2a674c80968d508eb9636439 -> Peer 0d424b3961bf4d859028eed7f3748eff (127.4.177.2:37655): Couldn't send request to peer 0d424b3961bf4d859028eed7f3748eff. Status: Network error: Client connection negotiation failed: client connection to 127.4.177.2:37655: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260812 06:37:33.446086 14247 tablet.cc:2406] T fcde898f35224c75ae069a036da1f9df P fd09b24d2a674c80968d508eb9636439: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 06:37:33.545120 14302 batcher.cc:441] Timed out: Failed to write batch of 50 ops to tablet fcde898f35224c75ae069a036da1f9df after 12 attempt(s): Failed to write to server: (no server available): Write(tablet: fcde898f35224c75ae069a036da1f9df, num_ops: 50, num_attempts: 12) passed its deadline: Remote error: Service unavailable: op on tablet fcde898f35224c75ae069a036da1f9df rejected due to memory pressure: the memory usage of this op (1502305) plus the current consumption (1502305) exceeds the op memory limit (2097152) or the limit of an ancestral memory tracker.
W20260812 06:37:33.546502 14302 meta_cache.cc:302] tablet fcde898f35224c75ae069a036da1f9df: replica fd09b24d2a674c80968d508eb9636439 (127.4.177.3:43793) has failed: Timed out: Write RPC to 127.4.177.3:43793 timed out after 0.147s (SENT)
I20260812 06:37:33.895231  4804 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskCnZu03/build/debug/bin/kudu with pid 14016
I20260812 06:37:33.913892  4804 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskCnZu03/build/debug/bin/kudu with pid 13404
2026-08-12T06:37:33Z chronyd exiting
[       OK ] RaftConsensusITest.TestMemoryRemainsConstantDespiteTwoDeadFollowers (1538 ms)
[ RUN      ] RaftConsensusITest.TestEvictAbandonedFollowers
I20260812 06:37:33.945377  4804 ts_itest-base.cc:115] Starting cluster with:
I20260812 06:37:33.945429  4804 ts_itest-base.cc:116] --------------
I20260812 06:37:33.945441  4804 ts_itest-base.cc:117] 3 tablet servers
I20260812 06:37:33.945453  4804 ts_itest-base.cc:118] 3 replicas per TS
I20260812 06:37:33.945466  4804 ts_itest-base.cc:119] --------------
2026-08-12T06:37:33Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-08-12T06:37:33Z Disabled control of system clock
I20260812 06:37:33.957450  4804 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskCnZu03/build/debug/bin/kudu
/tmp/dist-test-taskCnZu03/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestEvictAbandonedFollowers.1786516568546459-4804-0/raft_consensus-itest-cluster/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestEvictAbandonedFollowers.1786516568546459-4804-0/raft_consensus-itest-cluster/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestEvictAbandonedFollowers.1786516568546459-4804-0/raft_consensus-itest-cluster/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestEvictAbandonedFollowers.1786516568546459-4804-0/raft_consensus-itest-cluster/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.177.62:40381
--webserver_interface=127.4.177.62
--webserver_port=0
--builtin_ntp_servers=127.4.177.20:41617
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.177.62:40381
--master_add_server_when_underreplicated=false
--raft_prepare_replacement_before_eviction=false with env {}
W20260812 06:37:34.063946 14334 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:37:34.064168 14334 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:37:34.064208 14334 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 06:37:34.067549 14334 flags.cc:432] Enabled experimental flag: --raft_prepare_replacement_before_eviction=false
W20260812 06:37:34.067622 14334 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 06:37:34.067643 14334 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:37:34.067662 14334 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 06:37:34.067680 14334 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260812 06:37:34.073132 14334 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestEvictAbandonedFollowers.1786516568546459-4804-0/raft_consensus-itest-cluster/master-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.177.20:41617
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false
--fs_data_dirs=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestEvictAbandonedFollowers.1786516568546459-4804-0/raft_consensus-itest-cluster/master-0/data
--fs_wal_dir=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestEvictAbandonedFollowers.1786516568546459-4804-0/raft_consensus-itest-cluster/master-0/wal
--master_add_server_when_underreplicated=false
--ipki_ca_key_size=768
--master_addresses=127.4.177.62:40381
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.177.62:40381
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestEvictAbandonedFollowers.1786516568546459-4804-0/raft_consensus-itest-cluster/master-0/data/info.pb
--webserver_interface=127.4.177.62
--webserver_port=0
--never_fsync=true
--heap_profile_path=/tmp/kudu.14334
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false

Master server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 06:37:34.074401 14334 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:37:34.075565 14334 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 06:37:34.081327 14343 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:37:34.081331 14342 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:37:34.081573 14334 server_base.cc:1061] running on GCE node
W20260812 06:37:34.081333 14346 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:37:34.082170 14334 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:37:34.083222 14334 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:37:34.084384 14334 hybrid_clock.cc:648] HybridClock initialized: now 1786516654084370 us; error 28 us; skew 500 ppm
I20260812 06:37:34.086263 14334 webserver.cc:533] Webserver started at http://127.4.177.62:39679/ using document root <none> and password file <none>
I20260812 06:37:34.086853 14334 fs_manager.cc:362] Metadata directory not provided
I20260812 06:37:34.086922 14334 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:37:34.087157 14334 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:37:34.088734 14334 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestEvictAbandonedFollowers.1786516568546459-4804-0/raft_consensus-itest-cluster/master-0/data/instance:
uuid: "ec72d7e9a2b8426da9e5edd71e454909"
format_stamp: "Formatted at 2026-08-12 06:37:34 on dist-test-slave-jztv"
I20260812 06:37:34.089238 14334 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestEvictAbandonedFollowers.1786516568546459-4804-0/raft_consensus-itest-cluster/master-0/wal/instance:
uuid: "ec72d7e9a2b8426da9e5edd71e454909"
format_stamp: "Formatted at 2026-08-12 06:37:34 on dist-test-slave-jztv"
I20260812 06:37:34.092944 14334 fs_manager.cc:696] Time spent creating directory manager: real 0.003s	user 0.006s	sys 0.000s
I20260812 06:37:34.095366 14360 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 06:37:34.096446 14334 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260812 06:37:34.096583 14334 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestEvictAbandonedFollowers.1786516568546459-4804-0/raft_consensus-itest-cluster/master-0/data,/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestEvictAbandonedFollowers.1786516568546459-4804-0/raft_consensus-itest-cluster/master-0/wal
uuid: "ec72d7e9a2b8426da9e5edd71e454909"
format_stamp: "Formatted at 2026-08-12 06:37:34 on dist-test-slave-jztv"
I20260812 06:37:34.096688 14334 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestEvictAbandonedFollowers.1786516568546459-4804-0/raft_consensus-itest-cluster/master-0/wal
metadata directory: /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestEvictAbandonedFollowers.1786516568546459-4804-0/raft_consensus-itest-cluster/master-0/wal
1 data directories: /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestEvictAbandonedFollowers.1786516568546459-4804-0/raft_consensus-itest-cluster/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:37:34.118662 14334 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 06:37:34.119407 14334 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:37:34.119613 14334 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:37:34.127422 14334 rpc_server.cc:307] RPC server started. Bound to: 127.4.177.62:40381
I20260812 06:37:34.127439 14452 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.4.177.62:40381 every 8 connection(s)
I20260812 06:37:34.128564 14334 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestEvictAbandonedFollowers.1786516568546459-4804-0/raft_consensus-itest-cluster/master-0/data/info.pb
I20260812 06:37:34.131423 14455 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:37:34.134620  4804 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskCnZu03/build/debug/bin/kudu as pid 14334
I20260812 06:37:34.134783  4804 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestEvictAbandonedFollowers.1786516568546459-4804-0/raft_consensus-itest-cluster/master-0/wal/instance
I20260812 06:37:34.136987 14455 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ec72d7e9a2b8426da9e5edd71e454909: Bootstrap starting.
I20260812 06:37:34.139716 14455 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ec72d7e9a2b8426da9e5edd71e454909: Neither blocks nor log segments found. Creating new log.
I20260812 06:37:34.140553 14455 log.cc:826] T 00000000000000000000000000000000 P ec72d7e9a2b8426da9e5edd71e454909: Log is configured to *not* fsync() on all Append() calls
I20260812 06:37:34.142540 14455 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ec72d7e9a2b8426da9e5edd71e454909: No bootstrap required, opened a new log
I20260812 06:37:34.145385 14455 raft_consensus.cc:359] T 00000000000000000000000000000000 P ec72d7e9a2b8426da9e5edd71e454909 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec72d7e9a2b8426da9e5edd71e454909" member_type: VOTER last_known_addr { host: "127.4.177.62" port: 40381 } }
I20260812 06:37:34.145628 14455 raft_consensus.cc:385] T 00000000000000000000000000000000 P ec72d7e9a2b8426da9e5edd71e454909 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:37:34.145749 14455 raft_consensus.cc:740] T 00000000000000000000000000000000 P ec72d7e9a2b8426da9e5edd71e454909 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec72d7e9a2b8426da9e5edd71e454909, State: Initialized, Role: FOLLOWER
I20260812 06:37:34.146154 14455 consensus_queue.cc:260] T 00000000000000000000000000000000 P ec72d7e9a2b8426da9e5edd71e454909 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec72d7e9a2b8426da9e5edd71e454909" member_type: VOTER last_known_addr { host: "127.4.177.62" port: 40381 } }
I20260812 06:37:34.146363 14455 raft_consensus.cc:399] T 00000000000000000000000000000000 P ec72d7e9a2b8426da9e5edd71e454909 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 06:37:34.146428 14455 raft_consensus.cc:493] T 00000000000000000000000000000000 P ec72d7e9a2b8426da9e5edd71e454909 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 06:37:34.146534 14455 raft_consensus.cc:3060] T 00000000000000000000000000000000 P ec72d7e9a2b8426da9e5edd71e454909 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:37:34.147433 14455 raft_consensus.cc:515] T 00000000000000000000000000000000 P ec72d7e9a2b8426da9e5edd71e454909 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec72d7e9a2b8426da9e5edd71e454909" member_type: VOTER last_known_addr { host: "127.4.177.62" port: 40381 } }
I20260812 06:37:34.147770 14455 leader_election.cc:304] T 00000000000000000000000000000000 P ec72d7e9a2b8426da9e5edd71e454909 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ec72d7e9a2b8426da9e5edd71e454909; no voters: 
I20260812 06:37:34.148084 14455 leader_election.cc:290] T 00000000000000000000000000000000 P ec72d7e9a2b8426da9e5edd71e454909 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260812 06:37:34.148229 14462 raft_consensus.cc:2804] T 00000000000000000000000000000000 P ec72d7e9a2b8426da9e5edd71e454909 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 06:37:34.148471 14462 raft_consensus.cc:697] T 00000000000000000000000000000000 P ec72d7e9a2b8426da9e5edd71e454909 [term 1 LEADER]: Becoming Leader. State: Replica: ec72d7e9a2b8426da9e5edd71e454909, State: Running, Role: LEADER
I20260812 06:37:34.148754 14462 consensus_queue.cc:237] T 00000000000000000000000000000000 P ec72d7e9a2b8426da9e5edd71e454909 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec72d7e9a2b8426da9e5edd71e454909" member_type: VOTER last_known_addr { host: "127.4.177.62" port: 40381 } }
I20260812 06:37:34.149240 14455 sys_catalog.cc:565] T 00000000000000000000000000000000 P ec72d7e9a2b8426da9e5edd71e454909 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 06:37:34.150386 14463 sys_catalog.cc:455] T 00000000000000000000000000000000 P ec72d7e9a2b8426da9e5edd71e454909 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ec72d7e9a2b8426da9e5edd71e454909" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec72d7e9a2b8426da9e5edd71e454909" member_type: VOTER last_known_addr { host: "127.4.177.62" port: 40381 } } }
I20260812 06:37:34.150518 14463 sys_catalog.cc:458] T 00000000000000000000000000000000 P ec72d7e9a2b8426da9e5edd71e454909 [sys.catalog]: This master's current role is: LEADER
I20260812 06:37:34.150786 14467 sys_catalog.cc:455] T 00000000000000000000000000000000 P ec72d7e9a2b8426da9e5edd71e454909 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ec72d7e9a2b8426da9e5edd71e454909. Latest consensus state: current_term: 1 leader_uuid: "ec72d7e9a2b8426da9e5edd71e454909" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec72d7e9a2b8426da9e5edd71e454909" member_type: VOTER last_known_addr { host: "127.4.177.62" port: 40381 } } }
I20260812 06:37:34.150870 14467 sys_catalog.cc:458] T 00000000000000000000000000000000 P ec72d7e9a2b8426da9e5edd71e454909 [sys.catalog]: This master's current role is: LEADER
I20260812 06:37:34.151070 14480 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260812 06:37:34.153785 14480 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260812 06:37:34.158769 14480 catalog_manager.cc:1383] Generated new cluster ID: 69f819c5a4eb4a1bbadf69a62ce46fdf
I20260812 06:37:34.158846 14480 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260812 06:37:34.185019 14480 catalog_manager.cc:1406] Generated new certificate authority record
I20260812 06:37:34.185911 14480 catalog_manager.cc:1540] Loading token signing keys...
I20260812 06:37:34.190598 14480 catalog_manager.cc:6092] T 00000000000000000000000000000000 P ec72d7e9a2b8426da9e5edd71e454909: Generated new TSK 0
I20260812 06:37:34.191236 14480 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260812 06:37:34.194442  4804 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskCnZu03/build/debug/bin/kudu
/tmp/dist-test-taskCnZu03/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestEvictAbandonedFollowers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestEvictAbandonedFollowers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestEvictAbandonedFollowers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestEvictAbandonedFollowers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.177.1:0
--local_ip_for_outbound_sockets=127.4.177.1
--webserver_interface=127.4.177.1
--webserver_port=0
--tserver_master_addrs=127.4.177.62:40381
--builtin_ntp_servers=127.4.177.20:41617
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false
--log_compression_codec=no_compression
--log_cache_size_limit_mb=1
--log_segment_size_mb=1
--log_async_preallocate_segments=false
--log_min_segments_to_retain=1
--log_max_segments_to_retain=3
--maintenance_manager_polling_interval_ms=100
--log_target_replay_size_mb=1
--max_cell_size_bytes=1000000 with env {}
W20260812 06:37:34.306115 14491 flags.cc:432] Enabled unsafe flag: --max_cell_size_bytes=1000000
W20260812 06:37:34.306505 14491 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:37:34.306568 14491 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:37:34.306645 14491 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 06:37:34.309945 14491 flags.cc:432] Enabled experimental flag: --log_compression_codec=no_compression
W20260812 06:37:34.310029 14491 flags.cc:432] Enabled experimental flag: --log_max_segments_to_retain=3
W20260812 06:37:34.310076 14491 flags.cc:432] Enabled experimental flag: --raft_prepare_replacement_before_eviction=false
W20260812 06:37:34.310128 14491 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:37:34.310247 14491 flags.cc:432] Enabled experimental flag: --log_target_replay_size_mb=1
W20260812 06:37:34.310297 14491 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.177.1
I20260812 06:37:34.314200 14491 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestEvictAbandonedFollowers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.177.20:41617
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--max_cell_size_bytes=1000000
--log_compression_codec=no_compression
--log_max_segments_to_retain=3
--log_cache_size_limit_mb=1
--log_async_preallocate_segments=false
--log_segment_size_mb=1
--raft_prepare_replacement_before_eviction=false
--fs_data_dirs=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestEvictAbandonedFollowers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestEvictAbandonedFollowers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.177.1:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestEvictAbandonedFollowers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--webserver_interface=127.4.177.1
--webserver_port=0
--tserver_master_addrs=127.4.177.62:40381
--never_fsync=true
--heap_profile_path=/tmp/kudu.14491
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--log_target_replay_size_mb=1
--maintenance_manager_polling_interval_ms=100
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.177.1

Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 06:37:34.315320 14491 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:37:34.316416 14491 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 06:37:34.322831 14504 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:37:34.322844 14505 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:37:34.322840 14508 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:37:34.323387 14491 server_base.cc:1061] running on GCE node
I20260812 06:37:34.323890 14491 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:37:34.324506 14491 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:37:34.325692 14491 hybrid_clock.cc:648] HybridClock initialized: now 1786516654325668 us; error 33 us; skew 500 ppm
I20260812 06:37:34.327744 14491 webserver.cc:533] Webserver started at http://127.4.177.1:41397/ using document root <none> and password file <none>
I20260812 06:37:34.328384 14491 fs_manager.cc:362] Metadata directory not provided
I20260812 06:37:34.328498 14491 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:37:34.328697 14491 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:37:34.330650 14491 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestEvictAbandonedFollowers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-0/data/instance:
uuid: "fceb009f981a47659a0191d99601580d"
format_stamp: "Formatted at 2026-08-12 06:37:34 on dist-test-slave-jztv"
I20260812 06:37:34.331095 14491 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestEvictAbandonedFollowers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-0/wal/instance:
uuid: "fceb009f981a47659a0191d99601580d"
format_stamp: "Formatted at 2026-08-12 06:37:34 on dist-test-slave-jztv"
I20260812 06:37:34.334432 14491 fs_manager.cc:696] Time spent creating directory manager: real 0.003s	user 0.003s	sys 0.001s
I20260812 06:37:34.336730 14517 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 06:37:34.337702 14491 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260812 06:37:34.337826 14491 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestEvictAbandonedFollowers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-0/data,/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestEvictAbandonedFollowers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-0/wal
uuid: "fceb009f981a47659a0191d99601580d"
format_stamp: "Formatted at 2026-08-12 06:37:34 on dist-test-slave-jztv"
I20260812 06:37:34.337939 14491 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestEvictAbandonedFollowers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-0/wal
metadata directory: /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestEvictAbandonedFollowers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-0/wal
1 data directories: /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestEvictAbandonedFollowers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:37:34.350613 14491 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 06:37:34.351200 14491 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:37:34.351375 14491 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:37:34.351985 14491 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260812 06:37:34.352934 14491 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:37:34.353006 14491 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260812 06:37:34.353075 14491 ts_tablet_manager.cc:616] Registered 0 tablets
I20260812 06:37:34.353123 14491 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260812 06:37:34.362144 14491 rpc_server.cc:307] RPC server started. Bound to: 127.4.177.1:44201
I20260812 06:37:34.362185 14702 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.4.177.1:44201 every 8 connection(s)
I20260812 06:37:34.363127 14491 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestEvictAbandonedFollowers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-0/data/info.pb
I20260812 06:37:34.371924 14704 heartbeater.cc:344] Connected to a master server at 127.4.177.62:40381
I20260812 06:37:34.372181 14704 heartbeater.cc:461] Registering TS with master...
I20260812 06:37:34.372637 14704 heartbeater.cc:507] Master 127.4.177.62:40381 requested a full tablet report, sending...
I20260812 06:37:34.372820  4804 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskCnZu03/build/debug/bin/kudu as pid 14491
I20260812 06:37:34.372962  4804 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestEvictAbandonedFollowers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-0/wal/instance
I20260812 06:37:34.374183 14382 ts_manager.cc:194] Registered new tserver with Master: fceb009f981a47659a0191d99601580d (127.4.177.1:44201)
I20260812 06:37:34.375476  4804 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskCnZu03/build/debug/bin/kudu
/tmp/dist-test-taskCnZu03/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestEvictAbandonedFollowers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestEvictAbandonedFollowers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestEvictAbandonedFollowers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestEvictAbandonedFollowers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.177.2:0
--local_ip_for_outbound_sockets=127.4.177.2
--webserver_interface=127.4.177.2
--webserver_port=0
--tserver_master_addrs=127.4.177.62:40381
--builtin_ntp_servers=127.4.177.20:41617
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false
--log_compression_codec=no_compression
--log_cache_size_limit_mb=1
--log_segment_size_mb=1
--log_async_preallocate_segments=false
--log_min_segments_to_retain=1
--log_max_segments_to_retain=3
--maintenance_manager_polling_interval_ms=100
--log_target_replay_size_mb=1
--max_cell_size_bytes=1000000 with env {}
I20260812 06:37:34.375486 14382 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.4.177.1:48471
W20260812 06:37:34.484524 14707 flags.cc:432] Enabled unsafe flag: --max_cell_size_bytes=1000000
W20260812 06:37:34.484825 14707 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:37:34.484884 14707 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:37:34.484968 14707 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 06:37:34.488363 14707 flags.cc:432] Enabled experimental flag: --log_compression_codec=no_compression
W20260812 06:37:34.488444 14707 flags.cc:432] Enabled experimental flag: --log_max_segments_to_retain=3
W20260812 06:37:34.488490 14707 flags.cc:432] Enabled experimental flag: --raft_prepare_replacement_before_eviction=false
W20260812 06:37:34.488564 14707 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:37:34.488684 14707 flags.cc:432] Enabled experimental flag: --log_target_replay_size_mb=1
W20260812 06:37:34.488744 14707 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.177.2
I20260812 06:37:34.492763 14707 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestEvictAbandonedFollowers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-1/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.177.20:41617
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--max_cell_size_bytes=1000000
--log_compression_codec=no_compression
--log_max_segments_to_retain=3
--log_cache_size_limit_mb=1
--log_async_preallocate_segments=false
--log_segment_size_mb=1
--raft_prepare_replacement_before_eviction=false
--fs_data_dirs=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestEvictAbandonedFollowers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestEvictAbandonedFollowers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.177.2:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestEvictAbandonedFollowers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--webserver_interface=127.4.177.2
--webserver_port=0
--tserver_master_addrs=127.4.177.62:40381
--never_fsync=true
--heap_profile_path=/tmp/kudu.14707
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--log_target_replay_size_mb=1
--maintenance_manager_polling_interval_ms=100
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.177.2

Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 06:37:34.493902 14707 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:37:34.496090 14707 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 06:37:34.502709 14719 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:37:34.502736 14718 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:37:34.502768 14725 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:37:34.502972 14707 server_base.cc:1061] running on GCE node
I20260812 06:37:34.503357 14707 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:37:34.503906 14707 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:37:34.505105 14707 hybrid_clock.cc:648] HybridClock initialized: now 1786516654505098 us; error 40 us; skew 500 ppm
I20260812 06:37:34.506875 14707 webserver.cc:533] Webserver started at http://127.4.177.2:39267/ using document root <none> and password file <none>
I20260812 06:37:34.507449 14707 fs_manager.cc:362] Metadata directory not provided
I20260812 06:37:34.507535 14707 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:37:34.507756 14707 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:37:34.509298 14707 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestEvictAbandonedFollowers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-1/data/instance:
uuid: "6ff086d4092f4d3b963f1a1d9ac413d3"
format_stamp: "Formatted at 2026-08-12 06:37:34 on dist-test-slave-jztv"
I20260812 06:37:34.509804 14707 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestEvictAbandonedFollowers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-1/wal/instance:
uuid: "6ff086d4092f4d3b963f1a1d9ac413d3"
format_stamp: "Formatted at 2026-08-12 06:37:34 on dist-test-slave-jztv"
I20260812 06:37:34.513147 14707 fs_manager.cc:696] Time spent creating directory manager: real 0.003s	user 0.005s	sys 0.000s
I20260812 06:37:34.515308 14735 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 06:37:34.516312 14707 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.001s	sys 0.000s
I20260812 06:37:34.516474 14707 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestEvictAbandonedFollowers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-1/data,/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestEvictAbandonedFollowers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-1/wal
uuid: "6ff086d4092f4d3b963f1a1d9ac413d3"
format_stamp: "Formatted at 2026-08-12 06:37:34 on dist-test-slave-jztv"
I20260812 06:37:34.516594 14707 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestEvictAbandonedFollowers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-1/wal
metadata directory: /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestEvictAbandonedFollowers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-1/wal
1 data directories: /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestEvictAbandonedFollowers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:37:34.547111 14707 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 06:37:34.547817 14707 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:37:34.548022 14707 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:37:34.548646 14707 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260812 06:37:34.549700 14707 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:37:34.549774 14707 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260812 06:37:34.549847 14707 ts_tablet_manager.cc:616] Registered 0 tablets
I20260812 06:37:34.549906 14707 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260812 06:37:34.559919 14707 rpc_server.cc:307] RPC server started. Bound to: 127.4.177.2:37005
I20260812 06:37:34.559949 14929 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.4.177.2:37005 every 8 connection(s)
I20260812 06:37:34.560917 14707 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestEvictAbandonedFollowers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-1/data/info.pb
I20260812 06:37:34.564178  4804 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskCnZu03/build/debug/bin/kudu as pid 14707
I20260812 06:37:34.564338  4804 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestEvictAbandonedFollowers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-1/wal/instance
I20260812 06:37:34.566954  4804 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskCnZu03/build/debug/bin/kudu
/tmp/dist-test-taskCnZu03/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestEvictAbandonedFollowers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestEvictAbandonedFollowers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestEvictAbandonedFollowers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestEvictAbandonedFollowers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.177.3:0
--local_ip_for_outbound_sockets=127.4.177.3
--webserver_interface=127.4.177.3
--webserver_port=0
--tserver_master_addrs=127.4.177.62:40381
--builtin_ntp_servers=127.4.177.20:41617
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_prepare_replacement_before_eviction=false
--log_compression_codec=no_compression
--log_cache_size_limit_mb=1
--log_segment_size_mb=1
--log_async_preallocate_segments=false
--log_min_segments_to_retain=1
--log_max_segments_to_retain=3
--maintenance_manager_polling_interval_ms=100
--log_target_replay_size_mb=1
--max_cell_size_bytes=1000000 with env {}
I20260812 06:37:34.570521 14931 heartbeater.cc:344] Connected to a master server at 127.4.177.62:40381
I20260812 06:37:34.570787 14931 heartbeater.cc:461] Registering TS with master...
I20260812 06:37:34.571264 14931 heartbeater.cc:507] Master 127.4.177.62:40381 requested a full tablet report, sending...
I20260812 06:37:34.572326 14382 ts_manager.cc:194] Registered new tserver with Master: 6ff086d4092f4d3b963f1a1d9ac413d3 (127.4.177.2:37005)
I20260812 06:37:34.572929 14382 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.4.177.2:34207
W20260812 06:37:34.675976 14939 flags.cc:432] Enabled unsafe flag: --max_cell_size_bytes=1000000
W20260812 06:37:34.676275 14939 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:37:34.676334 14939 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:37:34.676411 14939 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 06:37:34.679804 14939 flags.cc:432] Enabled experimental flag: --log_compression_codec=no_compression
W20260812 06:37:34.679905 14939 flags.cc:432] Enabled experimental flag: --log_max_segments_to_retain=3
W20260812 06:37:34.679956 14939 flags.cc:432] Enabled experimental flag: --raft_prepare_replacement_before_eviction=false
W20260812 06:37:34.680032 14939 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:37:34.680156 14939 flags.cc:432] Enabled experimental flag: --log_target_replay_size_mb=1
W20260812 06:37:34.680215 14939 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.177.3
I20260812 06:37:34.684194 14939 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestEvictAbandonedFollowers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-2/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.177.20:41617
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--max_cell_size_bytes=1000000
--log_compression_codec=no_compression
--log_max_segments_to_retain=3
--log_cache_size_limit_mb=1
--log_async_preallocate_segments=false
--log_segment_size_mb=1
--raft_prepare_replacement_before_eviction=false
--fs_data_dirs=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestEvictAbandonedFollowers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestEvictAbandonedFollowers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.177.3:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestEvictAbandonedFollowers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--webserver_interface=127.4.177.3
--webserver_port=0
--tserver_master_addrs=127.4.177.62:40381
--never_fsync=true
--heap_profile_path=/tmp/kudu.14939
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--log_target_replay_size_mb=1
--maintenance_manager_polling_interval_ms=100
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.177.3

Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 06:37:34.685477 14939 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:37:34.686689 14939 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 06:37:34.693156 14947 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:37:34.693140 14946 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:37:34.693143 14951 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:37:34.693519 14939 server_base.cc:1061] running on GCE node
I20260812 06:37:34.693862 14939 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:37:34.694412 14939 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:37:34.695588 14939 hybrid_clock.cc:648] HybridClock initialized: now 1786516654695567 us; error 32 us; skew 500 ppm
I20260812 06:37:34.697290 14939 webserver.cc:533] Webserver started at http://127.4.177.3:39151/ using document root <none> and password file <none>
I20260812 06:37:34.697907 14939 fs_manager.cc:362] Metadata directory not provided
I20260812 06:37:34.697968 14939 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:37:34.698184 14939 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:37:34.699838 14939 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestEvictAbandonedFollowers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-2/data/instance:
uuid: "4b796b71507f4df8b877a17607df60c9"
format_stamp: "Formatted at 2026-08-12 06:37:34 on dist-test-slave-jztv"
I20260812 06:37:34.700325 14939 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestEvictAbandonedFollowers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-2/wal/instance:
uuid: "4b796b71507f4df8b877a17607df60c9"
format_stamp: "Formatted at 2026-08-12 06:37:34 on dist-test-slave-jztv"
I20260812 06:37:34.703742 14939 fs_manager.cc:696] Time spent creating directory manager: real 0.003s	user 0.001s	sys 0.003s
I20260812 06:37:34.705942 14966 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 06:37:34.706971 14939 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.001s
I20260812 06:37:34.707085 14939 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestEvictAbandonedFollowers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-2/data,/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestEvictAbandonedFollowers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-2/wal
uuid: "4b796b71507f4df8b877a17607df60c9"
format_stamp: "Formatted at 2026-08-12 06:37:34 on dist-test-slave-jztv"
I20260812 06:37:34.707180 14939 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestEvictAbandonedFollowers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-2/wal
metadata directory: /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestEvictAbandonedFollowers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-2/wal
1 data directories: /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestEvictAbandonedFollowers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:37:34.719794 14939 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 06:37:34.720359 14939 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:37:34.720547 14939 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:37:34.721184 14939 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260812 06:37:34.722200 14939 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:37:34.722301 14939 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260812 06:37:34.722383 14939 ts_tablet_manager.cc:616] Registered 0 tablets
I20260812 06:37:34.722419 14939 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260812 06:37:34.731678 14939 rpc_server.cc:307] RPC server started. Bound to: 127.4.177.3:46429
I20260812 06:37:34.731706 15177 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.4.177.3:46429 every 8 connection(s)
I20260812 06:37:34.732621 14939 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestEvictAbandonedFollowers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-2/data/info.pb
I20260812 06:37:34.735821  4804 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskCnZu03/build/debug/bin/kudu as pid 14939
I20260812 06:37:34.735973  4804 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestEvictAbandonedFollowers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-2/wal/instance
I20260812 06:37:34.741555 15178 heartbeater.cc:344] Connected to a master server at 127.4.177.62:40381
I20260812 06:37:34.741820 15178 heartbeater.cc:461] Registering TS with master...
I20260812 06:37:34.742349 15178 heartbeater.cc:507] Master 127.4.177.62:40381 requested a full tablet report, sending...
I20260812 06:37:34.743309 14382 ts_manager.cc:194] Registered new tserver with Master: 4b796b71507f4df8b877a17607df60c9 (127.4.177.3:46429)
I20260812 06:37:34.743867 14382 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.4.177.3:41577
I20260812 06:37:34.750041  4804 external_mini_cluster.cc:955] 3 TS(s) registered with all masters
I20260812 06:37:34.758975 14382 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33468:
name: "TestTable"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
owner: "alice"
I20260812 06:37:34.777660 15059 tablet_service.cc:1511] Processing CreateTablet for tablet e8828eabae374486b53b75b70d4bede8 (DEFAULT_TABLE table=TestTable [id=224dcaddf3a644f08521b6baca8510e8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 06:37:34.777665 14594 tablet_service.cc:1511] Processing CreateTablet for tablet e8828eabae374486b53b75b70d4bede8 (DEFAULT_TABLE table=TestTable [id=224dcaddf3a644f08521b6baca8510e8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 06:37:34.778291 14817 tablet_service.cc:1511] Processing CreateTablet for tablet e8828eabae374486b53b75b70d4bede8 (DEFAULT_TABLE table=TestTable [id=224dcaddf3a644f08521b6baca8510e8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 06:37:34.778604 15059 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e8828eabae374486b53b75b70d4bede8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:37:34.778635 14594 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e8828eabae374486b53b75b70d4bede8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:37:34.779090 14817 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e8828eabae374486b53b75b70d4bede8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:37:34.783600 15207 tablet_bootstrap.cc:492] T e8828eabae374486b53b75b70d4bede8 P 4b796b71507f4df8b877a17607df60c9: Bootstrap starting.
I20260812 06:37:34.784538 15209 tablet_bootstrap.cc:492] T e8828eabae374486b53b75b70d4bede8 P 6ff086d4092f4d3b963f1a1d9ac413d3: Bootstrap starting.
I20260812 06:37:34.786020 15207 tablet_bootstrap.cc:654] T e8828eabae374486b53b75b70d4bede8 P 4b796b71507f4df8b877a17607df60c9: Neither blocks nor log segments found. Creating new log.
I20260812 06:37:34.786454 15209 tablet_bootstrap.cc:654] T e8828eabae374486b53b75b70d4bede8 P 6ff086d4092f4d3b963f1a1d9ac413d3: Neither blocks nor log segments found. Creating new log.
I20260812 06:37:34.786559 15208 tablet_bootstrap.cc:492] T e8828eabae374486b53b75b70d4bede8 P fceb009f981a47659a0191d99601580d: Bootstrap starting.
I20260812 06:37:34.786863 15207 log.cc:826] T e8828eabae374486b53b75b70d4bede8 P 4b796b71507f4df8b877a17607df60c9: Log is configured to *not* fsync() on all Append() calls
I20260812 06:37:34.787235 15209 log.cc:826] T e8828eabae374486b53b75b70d4bede8 P 6ff086d4092f4d3b963f1a1d9ac413d3: Log is configured to *not* fsync() on all Append() calls
I20260812 06:37:34.788488 15207 tablet_bootstrap.cc:492] T e8828eabae374486b53b75b70d4bede8 P 4b796b71507f4df8b877a17607df60c9: No bootstrap required, opened a new log
I20260812 06:37:34.788565 15208 tablet_bootstrap.cc:654] T e8828eabae374486b53b75b70d4bede8 P fceb009f981a47659a0191d99601580d: Neither blocks nor log segments found. Creating new log.
I20260812 06:37:34.788708 15207 ts_tablet_manager.cc:1403] T e8828eabae374486b53b75b70d4bede8 P 4b796b71507f4df8b877a17607df60c9: Time spent bootstrapping tablet: real 0.005s	user 0.001s	sys 0.003s
I20260812 06:37:34.788946 15209 tablet_bootstrap.cc:492] T e8828eabae374486b53b75b70d4bede8 P 6ff086d4092f4d3b963f1a1d9ac413d3: No bootstrap required, opened a new log
I20260812 06:37:34.789343 15208 log.cc:826] T e8828eabae374486b53b75b70d4bede8 P fceb009f981a47659a0191d99601580d: Log is configured to *not* fsync() on all Append() calls
I20260812 06:37:34.789122 15209 ts_tablet_manager.cc:1403] T e8828eabae374486b53b75b70d4bede8 P 6ff086d4092f4d3b963f1a1d9ac413d3: Time spent bootstrapping tablet: real 0.005s	user 0.003s	sys 0.000s
I20260812 06:37:34.791432 15208 tablet_bootstrap.cc:492] T e8828eabae374486b53b75b70d4bede8 P fceb009f981a47659a0191d99601580d: No bootstrap required, opened a new log
I20260812 06:37:34.791607 15208 ts_tablet_manager.cc:1403] T e8828eabae374486b53b75b70d4bede8 P fceb009f981a47659a0191d99601580d: Time spent bootstrapping tablet: real 0.005s	user 0.003s	sys 0.001s
I20260812 06:37:34.792062 15207 raft_consensus.cc:359] T e8828eabae374486b53b75b70d4bede8 P 4b796b71507f4df8b877a17607df60c9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fceb009f981a47659a0191d99601580d" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 44201 } } peers { permanent_uuid: "4b796b71507f4df8b877a17607df60c9" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 46429 } } peers { permanent_uuid: "6ff086d4092f4d3b963f1a1d9ac413d3" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 37005 } }
I20260812 06:37:34.792266 15207 raft_consensus.cc:385] T e8828eabae374486b53b75b70d4bede8 P 4b796b71507f4df8b877a17607df60c9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:37:34.792333 15207 raft_consensus.cc:740] T e8828eabae374486b53b75b70d4bede8 P 4b796b71507f4df8b877a17607df60c9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b796b71507f4df8b877a17607df60c9, State: Initialized, Role: FOLLOWER
I20260812 06:37:34.792768 15207 consensus_queue.cc:260] T e8828eabae374486b53b75b70d4bede8 P 4b796b71507f4df8b877a17607df60c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fceb009f981a47659a0191d99601580d" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 44201 } } peers { permanent_uuid: "4b796b71507f4df8b877a17607df60c9" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 46429 } } peers { permanent_uuid: "6ff086d4092f4d3b963f1a1d9ac413d3" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 37005 } }
I20260812 06:37:34.793143 15209 raft_consensus.cc:359] T e8828eabae374486b53b75b70d4bede8 P 6ff086d4092f4d3b963f1a1d9ac413d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fceb009f981a47659a0191d99601580d" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 44201 } } peers { permanent_uuid: "4b796b71507f4df8b877a17607df60c9" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 46429 } } peers { permanent_uuid: "6ff086d4092f4d3b963f1a1d9ac413d3" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 37005 } }
I20260812 06:37:34.793337 15209 raft_consensus.cc:385] T e8828eabae374486b53b75b70d4bede8 P 6ff086d4092f4d3b963f1a1d9ac413d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:37:34.793397 15209 raft_consensus.cc:740] T e8828eabae374486b53b75b70d4bede8 P 6ff086d4092f4d3b963f1a1d9ac413d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6ff086d4092f4d3b963f1a1d9ac413d3, State: Initialized, Role: FOLLOWER
I20260812 06:37:34.793416 15178 heartbeater.cc:499] Master 127.4.177.62:40381 was elected leader, sending a full tablet report...
I20260812 06:37:34.793846 15209 consensus_queue.cc:260] T e8828eabae374486b53b75b70d4bede8 P 6ff086d4092f4d3b963f1a1d9ac413d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fceb009f981a47659a0191d99601580d" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 44201 } } peers { permanent_uuid: "4b796b71507f4df8b877a17607df60c9" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 46429 } } peers { permanent_uuid: "6ff086d4092f4d3b963f1a1d9ac413d3" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 37005 } }
I20260812 06:37:34.794512 14931 heartbeater.cc:499] Master 127.4.177.62:40381 was elected leader, sending a full tablet report...
I20260812 06:37:34.794718 15209 ts_tablet_manager.cc:1434] T e8828eabae374486b53b75b70d4bede8 P 6ff086d4092f4d3b963f1a1d9ac413d3: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260812 06:37:34.794906 15207 ts_tablet_manager.cc:1434] T e8828eabae374486b53b75b70d4bede8 P 4b796b71507f4df8b877a17607df60c9: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.002s
I20260812 06:37:34.795285 15208 raft_consensus.cc:359] T e8828eabae374486b53b75b70d4bede8 P fceb009f981a47659a0191d99601580d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fceb009f981a47659a0191d99601580d" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 44201 } } peers { permanent_uuid: "4b796b71507f4df8b877a17607df60c9" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 46429 } } peers { permanent_uuid: "6ff086d4092f4d3b963f1a1d9ac413d3" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 37005 } }
I20260812 06:37:34.795501 15208 raft_consensus.cc:385] T e8828eabae374486b53b75b70d4bede8 P fceb009f981a47659a0191d99601580d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:37:34.795557 15208 raft_consensus.cc:740] T e8828eabae374486b53b75b70d4bede8 P fceb009f981a47659a0191d99601580d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fceb009f981a47659a0191d99601580d, State: Initialized, Role: FOLLOWER
I20260812 06:37:34.795969 15208 consensus_queue.cc:260] T e8828eabae374486b53b75b70d4bede8 P fceb009f981a47659a0191d99601580d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fceb009f981a47659a0191d99601580d" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 44201 } } peers { permanent_uuid: "4b796b71507f4df8b877a17607df60c9" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 46429 } } peers { permanent_uuid: "6ff086d4092f4d3b963f1a1d9ac413d3" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 37005 } }
I20260812 06:37:34.796636 14704 heartbeater.cc:499] Master 127.4.177.62:40381 was elected leader, sending a full tablet report...
I20260812 06:37:34.796847 15208 ts_tablet_manager.cc:1434] T e8828eabae374486b53b75b70d4bede8 P fceb009f981a47659a0191d99601580d: Time spent starting tablet: real 0.005s	user 0.000s	sys 0.004s
W20260812 06:37:34.833604 15180 tablet.cc:2406] T e8828eabae374486b53b75b70d4bede8 P 4b796b71507f4df8b877a17607df60c9: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 06:37:34.862356 14932 tablet.cc:2406] T e8828eabae374486b53b75b70d4bede8 P 6ff086d4092f4d3b963f1a1d9ac413d3: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 06:37:34.864766 14705 tablet.cc:2406] T e8828eabae374486b53b75b70d4bede8 P fceb009f981a47659a0191d99601580d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 06:37:34.909439 15216 raft_consensus.cc:493] T e8828eabae374486b53b75b70d4bede8 P fceb009f981a47659a0191d99601580d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 06:37:34.909693 15216 raft_consensus.cc:515] T e8828eabae374486b53b75b70d4bede8 P fceb009f981a47659a0191d99601580d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fceb009f981a47659a0191d99601580d" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 44201 } } peers { permanent_uuid: "4b796b71507f4df8b877a17607df60c9" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 46429 } } peers { permanent_uuid: "6ff086d4092f4d3b963f1a1d9ac413d3" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 37005 } }
I20260812 06:37:34.910993 15216 leader_election.cc:290] T e8828eabae374486b53b75b70d4bede8 P fceb009f981a47659a0191d99601580d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4b796b71507f4df8b877a17607df60c9 (127.4.177.3:46429), 6ff086d4092f4d3b963f1a1d9ac413d3 (127.4.177.2:37005)
I20260812 06:37:34.915817 14854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8828eabae374486b53b75b70d4bede8" candidate_uuid: "fceb009f981a47659a0191d99601580d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ff086d4092f4d3b963f1a1d9ac413d3" is_pre_election: true
I20260812 06:37:34.915817 15096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8828eabae374486b53b75b70d4bede8" candidate_uuid: "fceb009f981a47659a0191d99601580d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b796b71507f4df8b877a17607df60c9" is_pre_election: true
I20260812 06:37:34.916137 15096 raft_consensus.cc:2468] T e8828eabae374486b53b75b70d4bede8 P 4b796b71507f4df8b877a17607df60c9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fceb009f981a47659a0191d99601580d in term 0.
I20260812 06:37:34.916138 14854 raft_consensus.cc:2468] T e8828eabae374486b53b75b70d4bede8 P 6ff086d4092f4d3b963f1a1d9ac413d3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fceb009f981a47659a0191d99601580d in term 0.
I20260812 06:37:34.916623 14519 leader_election.cc:304] T e8828eabae374486b53b75b70d4bede8 P fceb009f981a47659a0191d99601580d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ff086d4092f4d3b963f1a1d9ac413d3, fceb009f981a47659a0191d99601580d; no voters: 
I20260812 06:37:34.916886 15216 raft_consensus.cc:2804] T e8828eabae374486b53b75b70d4bede8 P fceb009f981a47659a0191d99601580d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 06:37:34.916985 15216 raft_consensus.cc:493] T e8828eabae374486b53b75b70d4bede8 P fceb009f981a47659a0191d99601580d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 06:37:34.917045 15216 raft_consensus.cc:3060] T e8828eabae374486b53b75b70d4bede8 P fceb009f981a47659a0191d99601580d [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:37:34.917913 15216 raft_consensus.cc:515] T e8828eabae374486b53b75b70d4bede8 P fceb009f981a47659a0191d99601580d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fceb009f981a47659a0191d99601580d" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 44201 } } peers { permanent_uuid: "4b796b71507f4df8b877a17607df60c9" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 46429 } } peers { permanent_uuid: "6ff086d4092f4d3b963f1a1d9ac413d3" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 37005 } }
I20260812 06:37:34.918398 15216 leader_election.cc:290] T e8828eabae374486b53b75b70d4bede8 P fceb009f981a47659a0191d99601580d [CANDIDATE]: Term 1 election: Requested vote from peers 4b796b71507f4df8b877a17607df60c9 (127.4.177.3:46429), 6ff086d4092f4d3b963f1a1d9ac413d3 (127.4.177.2:37005)
I20260812 06:37:34.918882 15096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8828eabae374486b53b75b70d4bede8" candidate_uuid: "fceb009f981a47659a0191d99601580d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b796b71507f4df8b877a17607df60c9"
I20260812 06:37:34.918918 14854 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8828eabae374486b53b75b70d4bede8" candidate_uuid: "fceb009f981a47659a0191d99601580d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ff086d4092f4d3b963f1a1d9ac413d3"
I20260812 06:37:34.919039 15096 raft_consensus.cc:3060] T e8828eabae374486b53b75b70d4bede8 P 4b796b71507f4df8b877a17607df60c9 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:37:34.919039 14854 raft_consensus.cc:3060] T e8828eabae374486b53b75b70d4bede8 P 6ff086d4092f4d3b963f1a1d9ac413d3 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:37:34.919925 14854 raft_consensus.cc:2468] T e8828eabae374486b53b75b70d4bede8 P 6ff086d4092f4d3b963f1a1d9ac413d3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fceb009f981a47659a0191d99601580d in term 1.
I20260812 06:37:34.920120 15096 raft_consensus.cc:2468] T e8828eabae374486b53b75b70d4bede8 P 4b796b71507f4df8b877a17607df60c9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fceb009f981a47659a0191d99601580d in term 1.
I20260812 06:37:34.920323 14519 leader_election.cc:304] T e8828eabae374486b53b75b70d4bede8 P fceb009f981a47659a0191d99601580d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ff086d4092f4d3b963f1a1d9ac413d3, fceb009f981a47659a0191d99601580d; no voters: 
I20260812 06:37:34.920547 15216 raft_consensus.cc:2804] T e8828eabae374486b53b75b70d4bede8 P fceb009f981a47659a0191d99601580d [term 1 FOLLOWER]: Leader election won for term 1
I20260812 06:37:34.920781 15216 raft_consensus.cc:697] T e8828eabae374486b53b75b70d4bede8 P fceb009f981a47659a0191d99601580d [term 1 LEADER]: Becoming Leader. State: Replica: fceb009f981a47659a0191d99601580d, State: Running, Role: LEADER
I20260812 06:37:34.921113 15216 consensus_queue.cc:237] T e8828eabae374486b53b75b70d4bede8 P fceb009f981a47659a0191d99601580d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fceb009f981a47659a0191d99601580d" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 44201 } } peers { permanent_uuid: "4b796b71507f4df8b877a17607df60c9" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 46429 } } peers { permanent_uuid: "6ff086d4092f4d3b963f1a1d9ac413d3" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 37005 } }
I20260812 06:37:34.924111 14382 catalog_manager.cc:5719] T e8828eabae374486b53b75b70d4bede8 P fceb009f981a47659a0191d99601580d reported cstate change: term changed from 0 to 1, leader changed from <none> to fceb009f981a47659a0191d99601580d (127.4.177.1). New cstate: current_term: 1 leader_uuid: "fceb009f981a47659a0191d99601580d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fceb009f981a47659a0191d99601580d" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 44201 } } peers { permanent_uuid: "4b796b71507f4df8b877a17607df60c9" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 46429 } } peers { permanent_uuid: "6ff086d4092f4d3b963f1a1d9ac413d3" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 37005 } } }
I20260812 06:37:34.959833  4804 external_mini_cluster.cc:955] 3 TS(s) registered with all masters
I20260812 06:37:34.961315  4804 ts_itest-base.cc:246] Waiting for 1 tablets on tserver fceb009f981a47659a0191d99601580d to finish bootstrapping
I20260812 06:37:34.967212  4804 ts_itest-base.cc:246] Waiting for 1 tablets on tserver 6ff086d4092f4d3b963f1a1d9ac413d3 to finish bootstrapping
I20260812 06:37:34.972802  4804 ts_itest-base.cc:246] Waiting for 1 tablets on tserver 4b796b71507f4df8b877a17607df60c9 to finish bootstrapping
I20260812 06:37:34.991096  4804 cluster_itest_util.cc:258] Not converged past 1 yet: 0.0 0.0 1.1
I20260812 06:37:35.094202  4804 cluster_itest_util.cc:258] Not converged past 1 yet: 0.0 0.0 1.1
I20260812 06:37:35.297416  4804 cluster_itest_util.cc:258] Not converged past 1 yet: 0.0 0.0 1.1
I20260812 06:37:35.542369 15234 consensus_queue.cc:1048] T e8828eabae374486b53b75b70d4bede8 P fceb009f981a47659a0191d99601580d [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ff086d4092f4d3b963f1a1d9ac413d3" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 37005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 06:37:35.544181 15235 consensus_queue.cc:1048] T e8828eabae374486b53b75b70d4bede8 P fceb009f981a47659a0191d99601580d [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b796b71507f4df8b877a17607df60c9" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 46429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 06:37:35.623234  4804 test_workload.cc:405] TestWorkload: Skipping table creation because table TestTable already exists
I20260812 06:37:35.623566  4804 raft_consensus-itest-base.cc:268] Waiting until we've written at least 4MB...
I20260812 06:37:35.719929  4804 raft_consensus-itest-base.cc:274] Waiting for log GC on fceb009f981a47659a0191d99601580d
I20260812 06:37:35.739353 15180 maintenance_manager.cc:419] P 4b796b71507f4df8b877a17607df60c9: Scheduling FlushMRSOp(e8828eabae374486b53b75b70d4bede8): 4076186 bytes log retention, and flush 6119689 bytes memory
I20260812 06:37:35.768323 14705 maintenance_manager.cc:419] P fceb009f981a47659a0191d99601580d: Scheduling FlushMRSOp(e8828eabae374486b53b75b70d4bede8): 4602240 bytes log retention, and flush 6119689 bytes memory
I20260812 06:37:35.801256 14974 maintenance_manager.cc:643] P 4b796b71507f4df8b877a17607df60c9: FlushMRSOp(e8828eabae374486b53b75b70d4bede8) complete. Timing: real 0.062s	user 0.061s	sys 0.000s Metrics: {"bytes_written":136001,"cfile_init":1,"compiler_manager_pool.queue_time_us":257,"drs_written":1,"lbm_read_time_us":74,"lbm_reads_lt_1ms":4,"lbm_write_time_us":704,"lbm_writes_lt_1ms":23,"peak_mem_usage":0,"rows_written":37,"thread_start_us":326,"threads_started":2}
I20260812 06:37:35.802425 15180 maintenance_manager.cc:419] P 4b796b71507f4df8b877a17607df60c9: Scheduling LogGCOp(e8828eabae374486b53b75b70d4bede8): free 2366879 bytes of WAL
I20260812 06:37:35.802752 14974 log_reader.cc:385] T e8828eabae374486b53b75b70d4bede8: removed 3 log segments from log reader
I20260812 06:37:35.802858 14974 log.cc:1079] T e8828eabae374486b53b75b70d4bede8 P 4b796b71507f4df8b877a17607df60c9: Deleting log segment in path: /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestEvictAbandonedFollowers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-2/wal/wals/e8828eabae374486b53b75b70d4bede8/wal-000000001 (ops 1-7)
I20260812 06:37:35.802965 14974 log.cc:1079] T e8828eabae374486b53b75b70d4bede8 P 4b796b71507f4df8b877a17607df60c9: Deleting log segment in path: /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestEvictAbandonedFollowers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-2/wal/wals/e8828eabae374486b53b75b70d4bede8/wal-000000002 (ops 8-12)
I20260812 06:37:35.803054 14974 log.cc:1079] T e8828eabae374486b53b75b70d4bede8 P 4b796b71507f4df8b877a17607df60c9: Deleting log segment in path: /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestEvictAbandonedFollowers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-2/wal/wals/e8828eabae374486b53b75b70d4bede8/wal-000000003 (ops 13-19)
I20260812 06:37:35.804037 14974 maintenance_manager.cc:643] P 4b796b71507f4df8b877a17607df60c9: LogGCOp(e8828eabae374486b53b75b70d4bede8) complete. Timing: real 0.001s	user 0.000s	sys 0.000s Metrics: {}
I20260812 06:37:35.804369 15180 maintenance_manager.cc:419] P 4b796b71507f4df8b877a17607df60c9: Scheduling UndoDeltaBlockGCOp(e8828eabae374486b53b75b70d4bede8): 550 bytes on disk
I20260812 06:37:35.805013 14974 maintenance_manager.cc:643] P 4b796b71507f4df8b877a17607df60c9: UndoDeltaBlockGCOp(e8828eabae374486b53b75b70d4bede8) complete. Timing: real 0.000s	user 0.000s	sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":67,"lbm_reads_lt_1ms":4}
I20260812 06:37:35.831207 14523 maintenance_manager.cc:643] P fceb009f981a47659a0191d99601580d: FlushMRSOp(e8828eabae374486b53b75b70d4bede8) complete. Timing: real 0.062s	user 0.058s	sys 0.004s Metrics: {"bytes_written":136001,"cfile_init":1,"compiler_manager_pool.queue_time_us":484,"dirs.queue_time_us":319,"dirs.run_cpu_time_us":209,"dirs.run_wall_time_us":927,"drs_written":1,"lbm_read_time_us":84,"lbm_reads_lt_1ms":4,"lbm_write_time_us":746,"lbm_writes_lt_1ms":23,"peak_mem_usage":0,"rows_written":37,"thread_start_us":343,"threads_started":2}
I20260812 06:37:35.832343 14705 maintenance_manager.cc:419] P fceb009f981a47659a0191d99601580d: Scheduling LogGCOp(e8828eabae374486b53b75b70d4bede8): free 2761290 bytes of WAL
I20260812 06:37:35.832804 14523 log_reader.cc:385] T e8828eabae374486b53b75b70d4bede8: removed 3 log segments from log reader
I20260812 06:37:35.832875 14523 log.cc:1079] T e8828eabae374486b53b75b70d4bede8 P fceb009f981a47659a0191d99601580d: Deleting log segment in path: /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestEvictAbandonedFollowers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-0/wal/wals/e8828eabae374486b53b75b70d4bede8/wal-000000001 (ops 1-8)
I20260812 06:37:35.832986 14523 log.cc:1079] T e8828eabae374486b53b75b70d4bede8 P fceb009f981a47659a0191d99601580d: Deleting log segment in path: /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestEvictAbandonedFollowers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-0/wal/wals/e8828eabae374486b53b75b70d4bede8/wal-000000002 (ops 9-15)
I20260812 06:37:35.833029 14523 log.cc:1079] T e8828eabae374486b53b75b70d4bede8 P fceb009f981a47659a0191d99601580d: Deleting log segment in path: /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestEvictAbandonedFollowers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-0/wal/wals/e8828eabae374486b53b75b70d4bede8/wal-000000003 (ops 16-22)
I20260812 06:37:35.834071 14523 maintenance_manager.cc:643] P fceb009f981a47659a0191d99601580d: LogGCOp(e8828eabae374486b53b75b70d4bede8) complete. Timing: real 0.001s	user 0.000s	sys 0.000s Metrics: {}
I20260812 06:37:35.834400 14705 maintenance_manager.cc:419] P fceb009f981a47659a0191d99601580d: Scheduling UndoDeltaBlockGCOp(e8828eabae374486b53b75b70d4bede8): 550 bytes on disk
I20260812 06:37:35.834995 14523 maintenance_manager.cc:643] P fceb009f981a47659a0191d99601580d: UndoDeltaBlockGCOp(e8828eabae374486b53b75b70d4bede8) complete. Timing: real 0.000s	user 0.000s	sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":69,"lbm_reads_lt_1ms":4}
I20260812 06:37:35.842914  4804 raft_consensus-itest-base.cc:279] Log GC complete on fceb009f981a47659a0191d99601580d
I20260812 06:37:37.844332  4804 raft_consensus-itest-base.cc:295] Servers converged with original term 1
I20260812 06:37:37.844403  4804 raft_consensus-itest-base.cc:300] Resuming 6ff086d4092f4d3b963f1a1d9ac413d3
I20260812 06:37:37.845674 15239 raft_consensus.cc:493] T e8828eabae374486b53b75b70d4bede8 P 6ff086d4092f4d3b963f1a1d9ac413d3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader fceb009f981a47659a0191d99601580d)
I20260812 06:37:37.845821 15239 raft_consensus.cc:515] T e8828eabae374486b53b75b70d4bede8 P 6ff086d4092f4d3b963f1a1d9ac413d3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fceb009f981a47659a0191d99601580d" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 44201 } } peers { permanent_uuid: "4b796b71507f4df8b877a17607df60c9" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 46429 } } peers { permanent_uuid: "6ff086d4092f4d3b963f1a1d9ac413d3" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 37005 } }
I20260812 06:37:37.847009 15239 leader_election.cc:290] T e8828eabae374486b53b75b70d4bede8 P 6ff086d4092f4d3b963f1a1d9ac413d3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers fceb009f981a47659a0191d99601580d (127.4.177.1:44201), 4b796b71507f4df8b877a17607df60c9 (127.4.177.3:46429)
I20260812 06:37:37.851518 15297 consensus_queue.cc:753] T e8828eabae374486b53b75b70d4bede8 P fceb009f981a47659a0191d99601580d [LEADER]: The logs necessary to catch up peer 6ff086d4092f4d3b963f1a1d9ac413d3 have been garbage collected. The follower will never be able to catch up (Not found: failed to read ops 3..32: Segment 1 which contained index 3 has been GCed)
I20260812 06:37:37.851897 15297 raft_consensus.cc:1012] T e8828eabae374486b53b75b70d4bede8 P fceb009f981a47659a0191d99601580d: Attempting to remove follower 6ff086d4092f4d3b963f1a1d9ac413d3 from the Raft config. Reason: The logs necessary to catch up peer 6ff086d4092f4d3b963f1a1d9ac413d3 have been garbage collected. The replica will never be able to catch up
I20260812 06:37:37.852784 15297 consensus_queue.cc:237] T e8828eabae374486b53b75b70d4bede8 P fceb009f981a47659a0191d99601580d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 38, Committed index: 38, Last appended: 1.38, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 39 OBSOLETE_local: false peers { permanent_uuid: "fceb009f981a47659a0191d99601580d" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 44201 } } peers { permanent_uuid: "4b796b71507f4df8b877a17607df60c9" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 46429 } }
I20260812 06:37:37.853523 14625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8828eabae374486b53b75b70d4bede8" candidate_uuid: "6ff086d4092f4d3b963f1a1d9ac413d3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "fceb009f981a47659a0191d99601580d" is_pre_election: true
I20260812 06:37:37.855113 15092 raft_consensus.cc:1275] T e8828eabae374486b53b75b70d4bede8 P 4b796b71507f4df8b877a17607df60c9 [term 1 FOLLOWER]: Refusing update from remote peer fceb009f981a47659a0191d99601580d: Log matching property violated. Preceding OpId in replica: term: 1 index: 38. Preceding OpId from leader: term: 1 index: 39. (index mismatch)
I20260812 06:37:37.855257 15096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8828eabae374486b53b75b70d4bede8" candidate_uuid: "6ff086d4092f4d3b963f1a1d9ac413d3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4b796b71507f4df8b877a17607df60c9" is_pre_election: true
I20260812 06:37:37.855623 15306 consensus_queue.cc:1048] T e8828eabae374486b53b75b70d4bede8 P fceb009f981a47659a0191d99601580d [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b796b71507f4df8b877a17607df60c9" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 46429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 39, Last known committed idx: 38, Time since last communication: 0.000s
I20260812 06:37:37.855692 14739 leader_election.cc:304] T e8828eabae374486b53b75b70d4bede8 P 6ff086d4092f4d3b963f1a1d9ac413d3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ff086d4092f4d3b963f1a1d9ac413d3; no voters: 4b796b71507f4df8b877a17607df60c9, fceb009f981a47659a0191d99601580d
I20260812 06:37:37.855989 15239 raft_consensus.cc:2749] T e8828eabae374486b53b75b70d4bede8 P 6ff086d4092f4d3b963f1a1d9ac413d3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 06:37:37.857849 15297 raft_consensus.cc:2955] T e8828eabae374486b53b75b70d4bede8 P fceb009f981a47659a0191d99601580d [term 1 LEADER]: Committing config change with OpId 1.39: config changed from index -1 to 39, VOTER 6ff086d4092f4d3b963f1a1d9ac413d3 (127.4.177.2) evicted. New config: { opid_index: 39 OBSOLETE_local: false peers { permanent_uuid: "fceb009f981a47659a0191d99601580d" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 44201 } } peers { permanent_uuid: "4b796b71507f4df8b877a17607df60c9" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 46429 } } }
I20260812 06:37:37.858649 15096 raft_consensus.cc:2955] T e8828eabae374486b53b75b70d4bede8 P 4b796b71507f4df8b877a17607df60c9 [term 1 FOLLOWER]: Committing config change with OpId 1.39: config changed from index -1 to 39, VOTER 6ff086d4092f4d3b963f1a1d9ac413d3 (127.4.177.2) evicted. New config: { opid_index: 39 OBSOLETE_local: false peers { permanent_uuid: "fceb009f981a47659a0191d99601580d" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 44201 } } peers { permanent_uuid: "4b796b71507f4df8b877a17607df60c9" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 46429 } } }
I20260812 06:37:37.860977 14388 catalog_manager.cc:5719] T e8828eabae374486b53b75b70d4bede8 P fceb009f981a47659a0191d99601580d reported cstate change: config changed from index -1 to 39, VOTER 6ff086d4092f4d3b963f1a1d9ac413d3 (127.4.177.2) evicted. New cstate: current_term: 1 leader_uuid: "fceb009f981a47659a0191d99601580d" committed_config { opid_index: 39 OBSOLETE_local: false peers { permanent_uuid: "fceb009f981a47659a0191d99601580d" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 44201 } } peers { permanent_uuid: "4b796b71507f4df8b877a17607df60c9" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 46429 } } }
I20260812 06:37:37.864750  4804 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskCnZu03/build/debug/bin/kudu with pid 14491
I20260812 06:37:37.869925 14817 tablet_service.cc:1558] Processing DeleteTablet for tablet e8828eabae374486b53b75b70d4bede8 with delete_type TABLET_DATA_TOMBSTONED (TS 6ff086d4092f4d3b963f1a1d9ac413d3 not found in new config with opid_index 39) from {username='slave'} at 127.0.0.1:38070
I20260812 06:37:37.871270 15319 tablet_replica.cc:333] T e8828eabae374486b53b75b70d4bede8 P 6ff086d4092f4d3b963f1a1d9ac413d3: stopping tablet replica
I20260812 06:37:37.871544 15319 raft_consensus.cc:2243] T e8828eabae374486b53b75b70d4bede8 P 6ff086d4092f4d3b963f1a1d9ac413d3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 06:37:37.871750 15319 pending_rounds.cc:70] T e8828eabae374486b53b75b70d4bede8 P 6ff086d4092f4d3b963f1a1d9ac413d3: Trying to abort 1 pending ops.
I20260812 06:37:37.871824 15319 pending_rounds.cc:77] T e8828eabae374486b53b75b70d4bede8 P 6ff086d4092f4d3b963f1a1d9ac413d3: Aborting op as it isn't in flight: id { term: 1 index: 2 } timestamp: 7317572221485883392 op_type: WRITE_OP write_request { tablet_id: "e8828eabae374486b53b75b70d4bede8" schema { columns { name: "key" type: INT32 is_key: true is_nullable: false immutable: false } columns { name: "int_val" type: INT32 is_key: false is_nullable: false immutable: false } columns { name: "string_val" type: STRING is_key: false is_nullable: true immutable: false } } row_operations { rows: "<redacted>""\001\007\000\251cf89\211\216!\000\000\000\000\000\000\000\000\000\000\002\000\000\000\000\000" indirect_data: "<redacted>""000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
I20260812 06:37:37.873749 15319 raft_consensus.cc:2272] T e8828eabae374486b53b75b70d4bede8 P 6ff086d4092f4d3b963f1a1d9ac413d3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 06:37:37.875077 15319 ts_tablet_manager.cc:1916] T e8828eabae374486b53b75b70d4bede8 P 6ff086d4092f4d3b963f1a1d9ac413d3: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260812 06:37:37.877161 15319 ts_tablet_manager.cc:1929] T e8828eabae374486b53b75b70d4bede8 P 6ff086d4092f4d3b963f1a1d9ac413d3: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.2
I20260812 06:37:37.877265 15319 log.cc:1199] T e8828eabae374486b53b75b70d4bede8 P 6ff086d4092f4d3b963f1a1d9ac413d3: Deleting WAL directory at /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestEvictAbandonedFollowers.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-1/wal/wals/e8828eabae374486b53b75b70d4bede8
I20260812 06:37:37.877919 14364 catalog_manager.cc:5050] TS 6ff086d4092f4d3b963f1a1d9ac413d3 (127.4.177.2:37005): tablet e8828eabae374486b53b75b70d4bede8 (table TestTable [id=224dcaddf3a644f08521b6baca8510e8]) successfully deleted
W20260812 06:37:37.880653 14366 connection.cc:573] server connection from 127.4.177.1:48471 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260812 06:37:37.880834  4804 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskCnZu03/build/debug/bin/kudu with pid 14707
I20260812 06:37:37.895459  4804 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskCnZu03/build/debug/bin/kudu with pid 14939
I20260812 06:37:37.910487  4804 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskCnZu03/build/debug/bin/kudu with pid 14334
2026-08-12T06:37:37Z chronyd exiting
[       OK ] RaftConsensusITest.TestEvictAbandonedFollowers (3994 ms)
[ RUN      ] RaftConsensusITest.TestElectionMetricsFailureDetectionDisabled
I20260812 06:37:37.939999  4804 ts_itest-base.cc:115] Starting cluster with:
I20260812 06:37:37.940056  4804 ts_itest-base.cc:116] --------------
I20260812 06:37:37.940068  4804 ts_itest-base.cc:117] 3 tablet servers
I20260812 06:37:37.940079  4804 ts_itest-base.cc:118] 3 replicas per TS
I20260812 06:37:37.940090  4804 ts_itest-base.cc:119] --------------
2026-08-12T06:37:37Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-08-12T06:37:37Z Disabled control of system clock
I20260812 06:37:37.952065  4804 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskCnZu03/build/debug/bin/kudu
/tmp/dist-test-taskCnZu03/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestElectionMetricsFailureDetectionDisabled.1786516568546459-4804-0/raft_consensus-itest-cluster/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestElectionMetricsFailureDetectionDisabled.1786516568546459-4804-0/raft_consensus-itest-cluster/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestElectionMetricsFailureDetectionDisabled.1786516568546459-4804-0/raft_consensus-itest-cluster/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestElectionMetricsFailureDetectionDisabled.1786516568546459-4804-0/raft_consensus-itest-cluster/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.177.62:43119
--webserver_interface=127.4.177.62
--webserver_port=0
--builtin_ntp_servers=127.4.177.20:38405
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.177.62:43119
--catalog_manager_wait_for_new_tablets_to_elect_leader=false with env {}
W20260812 06:37:38.061581 15332 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:37:38.061854 15332 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:37:38.061932 15332 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 06:37:38.065562 15332 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 06:37:38.065663 15332 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:37:38.065706 15332 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 06:37:38.065773 15332 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260812 06:37:38.069936 15332 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestElectionMetricsFailureDetectionDisabled.1786516568546459-4804-0/raft_consensus-itest-cluster/master-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.177.20:38405
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestElectionMetricsFailureDetectionDisabled.1786516568546459-4804-0/raft_consensus-itest-cluster/master-0/data
--fs_wal_dir=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestElectionMetricsFailureDetectionDisabled.1786516568546459-4804-0/raft_consensus-itest-cluster/master-0/wal
--catalog_manager_wait_for_new_tablets_to_elect_leader=false
--ipki_ca_key_size=768
--master_addresses=127.4.177.62:43119
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.177.62:43119
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestElectionMetricsFailureDetectionDisabled.1786516568546459-4804-0/raft_consensus-itest-cluster/master-0/data/info.pb
--webserver_interface=127.4.177.62
--webserver_port=0
--never_fsync=true
--heap_profile_path=/tmp/kudu.15332
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false

Master server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 06:37:38.071070 15332 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:37:38.072248 15332 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 06:37:38.077697 15346 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:37:38.077697 15341 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:37:38.077945 15339 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:37:38.078048 15332 server_base.cc:1061] running on GCE node
I20260812 06:37:38.078599 15332 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:37:38.079437 15332 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:37:38.080623 15332 hybrid_clock.cc:648] HybridClock initialized: now 1786516658080607 us; error 29 us; skew 500 ppm
I20260812 06:37:38.082363 15332 webserver.cc:533] Webserver started at http://127.4.177.62:41411/ using document root <none> and password file <none>
I20260812 06:37:38.082921 15332 fs_manager.cc:362] Metadata directory not provided
I20260812 06:37:38.083006 15332 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:37:38.083231 15332 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:37:38.084721 15332 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestElectionMetricsFailureDetectionDisabled.1786516568546459-4804-0/raft_consensus-itest-cluster/master-0/data/instance:
uuid: "bcc44b94dfda482cb8deec13f5404499"
format_stamp: "Formatted at 2026-08-12 06:37:38 on dist-test-slave-jztv"
I20260812 06:37:38.085201 15332 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestElectionMetricsFailureDetectionDisabled.1786516568546459-4804-0/raft_consensus-itest-cluster/master-0/wal/instance:
uuid: "bcc44b94dfda482cb8deec13f5404499"
format_stamp: "Formatted at 2026-08-12 06:37:38 on dist-test-slave-jztv"
I20260812 06:37:38.088428 15332 fs_manager.cc:696] Time spent creating directory manager: real 0.003s	user 0.001s	sys 0.004s
I20260812 06:37:38.090597 15352 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 06:37:38.091534 15332 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.001s	sys 0.001s
I20260812 06:37:38.091670 15332 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestElectionMetricsFailureDetectionDisabled.1786516568546459-4804-0/raft_consensus-itest-cluster/master-0/data,/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestElectionMetricsFailureDetectionDisabled.1786516568546459-4804-0/raft_consensus-itest-cluster/master-0/wal
uuid: "bcc44b94dfda482cb8deec13f5404499"
format_stamp: "Formatted at 2026-08-12 06:37:38 on dist-test-slave-jztv"
I20260812 06:37:38.091769 15332 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestElectionMetricsFailureDetectionDisabled.1786516568546459-4804-0/raft_consensus-itest-cluster/master-0/wal
metadata directory: /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestElectionMetricsFailureDetectionDisabled.1786516568546459-4804-0/raft_consensus-itest-cluster/master-0/wal
1 data directories: /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestElectionMetricsFailureDetectionDisabled.1786516568546459-4804-0/raft_consensus-itest-cluster/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:37:38.104507 15332 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 06:37:38.105144 15332 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:37:38.105340 15332 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:37:38.112816 15332 rpc_server.cc:307] RPC server started. Bound to: 127.4.177.62:43119
I20260812 06:37:38.112834 15430 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.4.177.62:43119 every 8 connection(s)
I20260812 06:37:38.113849 15332 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestElectionMetricsFailureDetectionDisabled.1786516568546459-4804-0/raft_consensus-itest-cluster/master-0/data/info.pb
I20260812 06:37:38.116909 15432 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:37:38.118752  4804 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskCnZu03/build/debug/bin/kudu as pid 15332
I20260812 06:37:38.118937  4804 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestElectionMetricsFailureDetectionDisabled.1786516568546459-4804-0/raft_consensus-itest-cluster/master-0/wal/instance
I20260812 06:37:38.122054 15432 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bcc44b94dfda482cb8deec13f5404499: Bootstrap starting.
I20260812 06:37:38.124696 15432 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bcc44b94dfda482cb8deec13f5404499: Neither blocks nor log segments found. Creating new log.
I20260812 06:37:38.125447 15432 log.cc:826] T 00000000000000000000000000000000 P bcc44b94dfda482cb8deec13f5404499: Log is configured to *not* fsync() on all Append() calls
I20260812 06:37:38.127158 15432 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bcc44b94dfda482cb8deec13f5404499: No bootstrap required, opened a new log
I20260812 06:37:38.129503 15432 raft_consensus.cc:359] T 00000000000000000000000000000000 P bcc44b94dfda482cb8deec13f5404499 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bcc44b94dfda482cb8deec13f5404499" member_type: VOTER last_known_addr { host: "127.4.177.62" port: 43119 } }
I20260812 06:37:38.129680 15432 raft_consensus.cc:385] T 00000000000000000000000000000000 P bcc44b94dfda482cb8deec13f5404499 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:37:38.129715 15432 raft_consensus.cc:740] T 00000000000000000000000000000000 P bcc44b94dfda482cb8deec13f5404499 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bcc44b94dfda482cb8deec13f5404499, State: Initialized, Role: FOLLOWER
I20260812 06:37:38.130147 15432 consensus_queue.cc:260] T 00000000000000000000000000000000 P bcc44b94dfda482cb8deec13f5404499 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bcc44b94dfda482cb8deec13f5404499" member_type: VOTER last_known_addr { host: "127.4.177.62" port: 43119 } }
I20260812 06:37:38.130329 15432 raft_consensus.cc:399] T 00000000000000000000000000000000 P bcc44b94dfda482cb8deec13f5404499 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 06:37:38.130373 15432 raft_consensus.cc:493] T 00000000000000000000000000000000 P bcc44b94dfda482cb8deec13f5404499 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 06:37:38.130481 15432 raft_consensus.cc:3060] T 00000000000000000000000000000000 P bcc44b94dfda482cb8deec13f5404499 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:37:38.131459 15432 raft_consensus.cc:515] T 00000000000000000000000000000000 P bcc44b94dfda482cb8deec13f5404499 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bcc44b94dfda482cb8deec13f5404499" member_type: VOTER last_known_addr { host: "127.4.177.62" port: 43119 } }
I20260812 06:37:38.131857 15432 leader_election.cc:304] T 00000000000000000000000000000000 P bcc44b94dfda482cb8deec13f5404499 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bcc44b94dfda482cb8deec13f5404499; no voters: 
I20260812 06:37:38.132194 15432 leader_election.cc:290] T 00000000000000000000000000000000 P bcc44b94dfda482cb8deec13f5404499 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260812 06:37:38.132337 15446 raft_consensus.cc:2804] T 00000000000000000000000000000000 P bcc44b94dfda482cb8deec13f5404499 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 06:37:38.132601 15446 raft_consensus.cc:697] T 00000000000000000000000000000000 P bcc44b94dfda482cb8deec13f5404499 [term 1 LEADER]: Becoming Leader. State: Replica: bcc44b94dfda482cb8deec13f5404499, State: Running, Role: LEADER
I20260812 06:37:38.132896 15446 consensus_queue.cc:237] T 00000000000000000000000000000000 P bcc44b94dfda482cb8deec13f5404499 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bcc44b94dfda482cb8deec13f5404499" member_type: VOTER last_known_addr { host: "127.4.177.62" port: 43119 } }
I20260812 06:37:38.133365 15432 sys_catalog.cc:565] T 00000000000000000000000000000000 P bcc44b94dfda482cb8deec13f5404499 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 06:37:38.134128 15448 sys_catalog.cc:455] T 00000000000000000000000000000000 P bcc44b94dfda482cb8deec13f5404499 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bcc44b94dfda482cb8deec13f5404499. Latest consensus state: current_term: 1 leader_uuid: "bcc44b94dfda482cb8deec13f5404499" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bcc44b94dfda482cb8deec13f5404499" member_type: VOTER last_known_addr { host: "127.4.177.62" port: 43119 } } }
I20260812 06:37:38.134280 15448 sys_catalog.cc:458] T 00000000000000000000000000000000 P bcc44b94dfda482cb8deec13f5404499 [sys.catalog]: This master's current role is: LEADER
I20260812 06:37:38.134658 15449 sys_catalog.cc:455] T 00000000000000000000000000000000 P bcc44b94dfda482cb8deec13f5404499 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bcc44b94dfda482cb8deec13f5404499" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bcc44b94dfda482cb8deec13f5404499" member_type: VOTER last_known_addr { host: "127.4.177.62" port: 43119 } } }
I20260812 06:37:38.134763 15449 sys_catalog.cc:458] T 00000000000000000000000000000000 P bcc44b94dfda482cb8deec13f5404499 [sys.catalog]: This master's current role is: LEADER
I20260812 06:37:38.135164 15458 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260812 06:37:38.138116 15458 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260812 06:37:38.143913 15458 catalog_manager.cc:1383] Generated new cluster ID: 325143532c32492c94f386e67fd7c8ab
I20260812 06:37:38.143981 15458 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260812 06:37:38.153769 15458 catalog_manager.cc:1406] Generated new certificate authority record
I20260812 06:37:38.154860 15458 catalog_manager.cc:1540] Loading token signing keys...
I20260812 06:37:38.166766 15458 catalog_manager.cc:6092] T 00000000000000000000000000000000 P bcc44b94dfda482cb8deec13f5404499: Generated new TSK 0
I20260812 06:37:38.167446 15458 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260812 06:37:38.176729  4804 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskCnZu03/build/debug/bin/kudu
/tmp/dist-test-taskCnZu03/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestElectionMetricsFailureDetectionDisabled.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestElectionMetricsFailureDetectionDisabled.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestElectionMetricsFailureDetectionDisabled.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestElectionMetricsFailureDetectionDisabled.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.177.1:0
--local_ip_for_outbound_sockets=127.4.177.1
--webserver_interface=127.4.177.1
--webserver_port=0
--tserver_master_addrs=127.4.177.62:43119
--builtin_ntp_servers=127.4.177.20:38405
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_heartbeat_interval_ms=100
--enable_leader_failure_detection=false with env {}
W20260812 06:37:38.288025 15473 flags.cc:432] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260812 06:37:38.288321 15473 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:37:38.288349 15473 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:37:38.288404 15473 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 06:37:38.292976 15473 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:37:38.293129 15473 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.177.1
I20260812 06:37:38.298550 15473 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestElectionMetricsFailureDetectionDisabled.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.177.20:38405
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--raft_heartbeat_interval_ms=100
--fs_data_dirs=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestElectionMetricsFailureDetectionDisabled.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestElectionMetricsFailureDetectionDisabled.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.177.1:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestElectionMetricsFailureDetectionDisabled.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--webserver_interface=127.4.177.1
--webserver_port=0
--tserver_master_addrs=127.4.177.62:43119
--never_fsync=true
--heap_profile_path=/tmp/kudu.15473
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.177.1

Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 06:37:38.300076 15473 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:37:38.301409 15473 file_cache.cc:504] Constructed file cache file cache with capacity 419430
E20260812 06:37:38.304246 15473 multi_raft_batcher.cc:235] multi_raft_heartbeat_window_ms should not be more than  raft_heartbeat_interval_ms / 2. , forcing multi_raft_heartbeat_window_ms = 50
W20260812 06:37:38.307976 15490 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:37:38.307986 15482 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:37:38.308251 15473 server_base.cc:1061] running on GCE node
W20260812 06:37:38.308444 15486 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:37:38.308825 15473 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:37:38.309403 15473 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:37:38.310588 15473 hybrid_clock.cc:648] HybridClock initialized: now 1786516658310565 us; error 38 us; skew 500 ppm
I20260812 06:37:38.312326 15473 webserver.cc:533] Webserver started at http://127.4.177.1:38047/ using document root <none> and password file <none>
I20260812 06:37:38.312934 15473 fs_manager.cc:362] Metadata directory not provided
I20260812 06:37:38.313020 15473 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:37:38.313253 15473 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:37:38.314870 15473 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestElectionMetricsFailureDetectionDisabled.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-0/data/instance:
uuid: "69b57269cd3e42c5bdc16a53a4e89240"
format_stamp: "Formatted at 2026-08-12 06:37:38 on dist-test-slave-jztv"
I20260812 06:37:38.315372 15473 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestElectionMetricsFailureDetectionDisabled.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-0/wal/instance:
uuid: "69b57269cd3e42c5bdc16a53a4e89240"
format_stamp: "Formatted at 2026-08-12 06:37:38 on dist-test-slave-jztv"
I20260812 06:37:38.318887 15473 fs_manager.cc:696] Time spent creating directory manager: real 0.003s	user 0.003s	sys 0.000s
I20260812 06:37:38.321146 15498 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 06:37:38.322101 15473 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.001s
I20260812 06:37:38.322258 15473 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestElectionMetricsFailureDetectionDisabled.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-0/data,/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestElectionMetricsFailureDetectionDisabled.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-0/wal
uuid: "69b57269cd3e42c5bdc16a53a4e89240"
format_stamp: "Formatted at 2026-08-12 06:37:38 on dist-test-slave-jztv"
I20260812 06:37:38.322376 15473 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestElectionMetricsFailureDetectionDisabled.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-0/wal
metadata directory: /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestElectionMetricsFailureDetectionDisabled.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-0/wal
1 data directories: /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestElectionMetricsFailureDetectionDisabled.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:37:38.333209 15473 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 06:37:38.333868 15473 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:37:38.334086 15473 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:37:38.334682 15473 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260812 06:37:38.335662 15473 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:37:38.335717 15473 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260812 06:37:38.335755 15473 ts_tablet_manager.cc:616] Registered 0 tablets
I20260812 06:37:38.335779 15473 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260812 06:37:38.345141 15473 rpc_server.cc:307] RPC server started. Bound to: 127.4.177.1:37009
I20260812 06:37:38.345211 15681 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.4.177.1:37009 every 8 connection(s)
I20260812 06:37:38.346298 15473 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestElectionMetricsFailureDetectionDisabled.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-0/data/info.pb
I20260812 06:37:38.354135  4804 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskCnZu03/build/debug/bin/kudu as pid 15473
I20260812 06:37:38.354313  4804 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestElectionMetricsFailureDetectionDisabled.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-0/wal/instance
I20260812 06:37:38.355377 15682 heartbeater.cc:344] Connected to a master server at 127.4.177.62:43119
I20260812 06:37:38.355611 15682 heartbeater.cc:461] Registering TS with master...
I20260812 06:37:38.356191 15682 heartbeater.cc:507] Master 127.4.177.62:43119 requested a full tablet report, sending...
I20260812 06:37:38.356700  4804 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskCnZu03/build/debug/bin/kudu
/tmp/dist-test-taskCnZu03/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestElectionMetricsFailureDetectionDisabled.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestElectionMetricsFailureDetectionDisabled.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestElectionMetricsFailureDetectionDisabled.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestElectionMetricsFailureDetectionDisabled.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.177.2:0
--local_ip_for_outbound_sockets=127.4.177.2
--webserver_interface=127.4.177.2
--webserver_port=0
--tserver_master_addrs=127.4.177.62:43119
--builtin_ntp_servers=127.4.177.20:38405
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_heartbeat_interval_ms=100
--enable_leader_failure_detection=false with env {}
I20260812 06:37:38.357746 15377 ts_manager.cc:194] Registered new tserver with Master: 69b57269cd3e42c5bdc16a53a4e89240 (127.4.177.1:37009)
I20260812 06:37:38.359359 15377 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.4.177.1:57485
W20260812 06:37:38.463897 15687 flags.cc:432] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260812 06:37:38.464100 15687 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:37:38.464123 15687 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:37:38.464160 15687 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 06:37:38.467988 15687 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:37:38.468120 15687 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.177.2
I20260812 06:37:38.471846 15687 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestElectionMetricsFailureDetectionDisabled.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-1/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.177.20:38405
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--raft_heartbeat_interval_ms=100
--fs_data_dirs=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestElectionMetricsFailureDetectionDisabled.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestElectionMetricsFailureDetectionDisabled.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.177.2:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestElectionMetricsFailureDetectionDisabled.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--webserver_interface=127.4.177.2
--webserver_port=0
--tserver_master_addrs=127.4.177.62:43119
--never_fsync=true
--heap_profile_path=/tmp/kudu.15687
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.177.2

Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 06:37:38.472875 15687 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:37:38.473949 15687 file_cache.cc:504] Constructed file cache file cache with capacity 419430
E20260812 06:37:38.476626 15687 multi_raft_batcher.cc:235] multi_raft_heartbeat_window_ms should not be more than  raft_heartbeat_interval_ms / 2. , forcing multi_raft_heartbeat_window_ms = 50
W20260812 06:37:38.480417 15706 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:37:38.480413 15703 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:37:38.480417 15699 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:37:38.480721 15687 server_base.cc:1061] running on GCE node
I20260812 06:37:38.481128 15687 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:37:38.481657 15687 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:37:38.482834 15687 hybrid_clock.cc:648] HybridClock initialized: now 1786516658482824 us; error 33 us; skew 500 ppm
I20260812 06:37:38.484479 15687 webserver.cc:533] Webserver started at http://127.4.177.2:44539/ using document root <none> and password file <none>
I20260812 06:37:38.485040 15687 fs_manager.cc:362] Metadata directory not provided
I20260812 06:37:38.485122 15687 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:37:38.485345 15687 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:37:38.486896 15687 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestElectionMetricsFailureDetectionDisabled.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-1/data/instance:
uuid: "613405f4f5894c14a690f9cb7dca93de"
format_stamp: "Formatted at 2026-08-12 06:37:38 on dist-test-slave-jztv"
I20260812 06:37:38.487370 15687 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestElectionMetricsFailureDetectionDisabled.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-1/wal/instance:
uuid: "613405f4f5894c14a690f9cb7dca93de"
format_stamp: "Formatted at 2026-08-12 06:37:38 on dist-test-slave-jztv"
I20260812 06:37:38.490568 15687 fs_manager.cc:696] Time spent creating directory manager: real 0.003s	user 0.003s	sys 0.000s
I20260812 06:37:38.492652 15712 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 06:37:38.493659 15687 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260812 06:37:38.493791 15687 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestElectionMetricsFailureDetectionDisabled.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-1/data,/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestElectionMetricsFailureDetectionDisabled.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-1/wal
uuid: "613405f4f5894c14a690f9cb7dca93de"
format_stamp: "Formatted at 2026-08-12 06:37:38 on dist-test-slave-jztv"
I20260812 06:37:38.493896 15687 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestElectionMetricsFailureDetectionDisabled.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-1/wal
metadata directory: /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestElectionMetricsFailureDetectionDisabled.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-1/wal
1 data directories: /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestElectionMetricsFailureDetectionDisabled.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:37:38.511982 15687 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 06:37:38.512614 15687 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:37:38.512804 15687 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:37:38.513358 15687 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260812 06:37:38.514380 15687 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:37:38.514448 15687 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260812 06:37:38.514525 15687 ts_tablet_manager.cc:616] Registered 0 tablets
I20260812 06:37:38.514567 15687 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260812 06:37:38.523780 15687 rpc_server.cc:307] RPC server started. Bound to: 127.4.177.2:35913
I20260812 06:37:38.523806 15899 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.4.177.2:35913 every 8 connection(s)
I20260812 06:37:38.524712 15687 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestElectionMetricsFailureDetectionDisabled.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-1/data/info.pb
I20260812 06:37:38.525338  4804 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskCnZu03/build/debug/bin/kudu as pid 15687
I20260812 06:37:38.525471  4804 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestElectionMetricsFailureDetectionDisabled.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-1/wal/instance
I20260812 06:37:38.527905  4804 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskCnZu03/build/debug/bin/kudu
/tmp/dist-test-taskCnZu03/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestElectionMetricsFailureDetectionDisabled.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestElectionMetricsFailureDetectionDisabled.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestElectionMetricsFailureDetectionDisabled.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestElectionMetricsFailureDetectionDisabled.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.177.3:0
--local_ip_for_outbound_sockets=127.4.177.3
--webserver_interface=127.4.177.3
--webserver_port=0
--tserver_master_addrs=127.4.177.62:43119
--builtin_ntp_servers=127.4.177.20:38405
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_heartbeat_interval_ms=100
--enable_leader_failure_detection=false with env {}
I20260812 06:37:38.534058 15902 heartbeater.cc:344] Connected to a master server at 127.4.177.62:43119
I20260812 06:37:38.534420 15902 heartbeater.cc:461] Registering TS with master...
I20260812 06:37:38.535046 15902 heartbeater.cc:507] Master 127.4.177.62:43119 requested a full tablet report, sending...
I20260812 06:37:38.536342 15377 ts_manager.cc:194] Registered new tserver with Master: 613405f4f5894c14a690f9cb7dca93de (127.4.177.2:35913)
I20260812 06:37:38.537207 15377 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.4.177.2:44635
W20260812 06:37:38.636937 15905 flags.cc:432] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260812 06:37:38.637185 15905 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:37:38.637210 15905 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:37:38.637254 15905 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 06:37:38.641214 15905 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:37:38.641333 15905 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.177.3
I20260812 06:37:38.645593 15905 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestElectionMetricsFailureDetectionDisabled.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-2/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.177.20:38405
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--raft_heartbeat_interval_ms=100
--fs_data_dirs=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestElectionMetricsFailureDetectionDisabled.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestElectionMetricsFailureDetectionDisabled.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.177.3:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestElectionMetricsFailureDetectionDisabled.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--webserver_interface=127.4.177.3
--webserver_port=0
--tserver_master_addrs=127.4.177.62:43119
--never_fsync=true
--heap_profile_path=/tmp/kudu.15905
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.177.3

Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 06:37:38.646883 15905 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:37:38.647989 15905 file_cache.cc:504] Constructed file cache file cache with capacity 419430
E20260812 06:37:38.651454 15905 multi_raft_batcher.cc:235] multi_raft_heartbeat_window_ms should not be more than  raft_heartbeat_interval_ms / 2. , forcing multi_raft_heartbeat_window_ms = 50
W20260812 06:37:38.655884 15912 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:37:38.655893 15916 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:37:38.655884 15913 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:37:38.656258 15905 server_base.cc:1061] running on GCE node
I20260812 06:37:38.656810 15905 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:37:38.657444 15905 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:37:38.658633 15905 hybrid_clock.cc:648] HybridClock initialized: now 1786516658658625 us; error 35 us; skew 500 ppm
I20260812 06:37:38.660377 15905 webserver.cc:533] Webserver started at http://127.4.177.3:40827/ using document root <none> and password file <none>
I20260812 06:37:38.660961 15905 fs_manager.cc:362] Metadata directory not provided
I20260812 06:37:38.661046 15905 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:37:38.661257 15905 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:37:38.662843 15905 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestElectionMetricsFailureDetectionDisabled.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-2/data/instance:
uuid: "d483e6d3aaf142f195a69b47f375a17f"
format_stamp: "Formatted at 2026-08-12 06:37:38 on dist-test-slave-jztv"
I20260812 06:37:38.663331 15905 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestElectionMetricsFailureDetectionDisabled.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-2/wal/instance:
uuid: "d483e6d3aaf142f195a69b47f375a17f"
format_stamp: "Formatted at 2026-08-12 06:37:38 on dist-test-slave-jztv"
I20260812 06:37:38.666676 15905 fs_manager.cc:696] Time spent creating directory manager: real 0.003s	user 0.003s	sys 0.000s
I20260812 06:37:38.668900 15932 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 06:37:38.669883 15905 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260812 06:37:38.670012 15905 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestElectionMetricsFailureDetectionDisabled.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-2/data,/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestElectionMetricsFailureDetectionDisabled.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-2/wal
uuid: "d483e6d3aaf142f195a69b47f375a17f"
format_stamp: "Formatted at 2026-08-12 06:37:38 on dist-test-slave-jztv"
I20260812 06:37:38.670112 15905 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestElectionMetricsFailureDetectionDisabled.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-2/wal
metadata directory: /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestElectionMetricsFailureDetectionDisabled.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-2/wal
1 data directories: /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestElectionMetricsFailureDetectionDisabled.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:37:38.678902 15905 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 06:37:38.679497 15905 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:37:38.679687 15905 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:37:38.680245 15905 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260812 06:37:38.681208 15905 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:37:38.681257 15905 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260812 06:37:38.681322 15905 ts_tablet_manager.cc:616] Registered 0 tablets
I20260812 06:37:38.681363 15905 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260812 06:37:38.690490 15905 rpc_server.cc:307] RPC server started. Bound to: 127.4.177.3:41823
I20260812 06:37:38.690559 16133 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.4.177.3:41823 every 8 connection(s)
I20260812 06:37:38.691493 15905 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestElectionMetricsFailureDetectionDisabled.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-2/data/info.pb
I20260812 06:37:38.696688  4804 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskCnZu03/build/debug/bin/kudu as pid 15905
I20260812 06:37:38.696828  4804 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.RaftConsensusITest.TestElectionMetricsFailureDetectionDisabled.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-2/wal/instance
I20260812 06:37:38.700882 16135 heartbeater.cc:344] Connected to a master server at 127.4.177.62:43119
I20260812 06:37:38.701144 16135 heartbeater.cc:461] Registering TS with master...
I20260812 06:37:38.701658 16135 heartbeater.cc:507] Master 127.4.177.62:43119 requested a full tablet report, sending...
I20260812 06:37:38.702721 15377 ts_manager.cc:194] Registered new tserver with Master: d483e6d3aaf142f195a69b47f375a17f (127.4.177.3:41823)
I20260812 06:37:38.703274 15377 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.4.177.3:58573
I20260812 06:37:38.711333  4804 external_mini_cluster.cc:955] 3 TS(s) registered with all masters
I20260812 06:37:38.720413 15377 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60926:
name: "TestTable"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
owner: "alice"
W20260812 06:37:38.723045 15377 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table TestTable in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 06:37:38.739296 16015 tablet_service.cc:1511] Processing CreateTablet for tablet 52ce1901589347d392b01d00179b4d68 (DEFAULT_TABLE table=TestTable [id=81ba0a3f886645479d0aaf38752aeeea]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 06:37:38.739625 15795 tablet_service.cc:1511] Processing CreateTablet for tablet 52ce1901589347d392b01d00179b4d68 (DEFAULT_TABLE table=TestTable [id=81ba0a3f886645479d0aaf38752aeeea]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 06:37:38.739571 15573 tablet_service.cc:1511] Processing CreateTablet for tablet 52ce1901589347d392b01d00179b4d68 (DEFAULT_TABLE table=TestTable [id=81ba0a3f886645479d0aaf38752aeeea]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 06:37:38.740428 16015 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 52ce1901589347d392b01d00179b4d68. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:37:38.740507 15795 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 52ce1901589347d392b01d00179b4d68. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:37:38.740507 15573 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 52ce1901589347d392b01d00179b4d68. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:37:38.745283 16164 tablet_bootstrap.cc:492] T 52ce1901589347d392b01d00179b4d68 P d483e6d3aaf142f195a69b47f375a17f: Bootstrap starting.
I20260812 06:37:38.745889 16168 tablet_bootstrap.cc:492] T 52ce1901589347d392b01d00179b4d68 P 69b57269cd3e42c5bdc16a53a4e89240: Bootstrap starting.
I20260812 06:37:38.746663 16169 tablet_bootstrap.cc:492] T 52ce1901589347d392b01d00179b4d68 P 613405f4f5894c14a690f9cb7dca93de: Bootstrap starting.
I20260812 06:37:38.747895 16164 tablet_bootstrap.cc:654] T 52ce1901589347d392b01d00179b4d68 P d483e6d3aaf142f195a69b47f375a17f: Neither blocks nor log segments found. Creating new log.
I20260812 06:37:38.747893 16168 tablet_bootstrap.cc:654] T 52ce1901589347d392b01d00179b4d68 P 69b57269cd3e42c5bdc16a53a4e89240: Neither blocks nor log segments found. Creating new log.
I20260812 06:37:38.748553 16169 tablet_bootstrap.cc:654] T 52ce1901589347d392b01d00179b4d68 P 613405f4f5894c14a690f9cb7dca93de: Neither blocks nor log segments found. Creating new log.
I20260812 06:37:38.748754 16164 log.cc:826] T 52ce1901589347d392b01d00179b4d68 P d483e6d3aaf142f195a69b47f375a17f: Log is configured to *not* fsync() on all Append() calls
I20260812 06:37:38.748754 16168 log.cc:826] T 52ce1901589347d392b01d00179b4d68 P 69b57269cd3e42c5bdc16a53a4e89240: Log is configured to *not* fsync() on all Append() calls
I20260812 06:37:38.749348 16169 log.cc:826] T 52ce1901589347d392b01d00179b4d68 P 613405f4f5894c14a690f9cb7dca93de: Log is configured to *not* fsync() on all Append() calls
I20260812 06:37:38.750470 16164 tablet_bootstrap.cc:492] T 52ce1901589347d392b01d00179b4d68 P d483e6d3aaf142f195a69b47f375a17f: No bootstrap required, opened a new log
I20260812 06:37:38.750470 16168 tablet_bootstrap.cc:492] T 52ce1901589347d392b01d00179b4d68 P 69b57269cd3e42c5bdc16a53a4e89240: No bootstrap required, opened a new log
I20260812 06:37:38.750650 16168 ts_tablet_manager.cc:1403] T 52ce1901589347d392b01d00179b4d68 P 69b57269cd3e42c5bdc16a53a4e89240: Time spent bootstrapping tablet: real 0.005s	user 0.004s	sys 0.000s
I20260812 06:37:38.750650 16164 ts_tablet_manager.cc:1403] T 52ce1901589347d392b01d00179b4d68 P d483e6d3aaf142f195a69b47f375a17f: Time spent bootstrapping tablet: real 0.006s	user 0.001s	sys 0.004s
I20260812 06:37:38.750898 16169 tablet_bootstrap.cc:492] T 52ce1901589347d392b01d00179b4d68 P 613405f4f5894c14a690f9cb7dca93de: No bootstrap required, opened a new log
I20260812 06:37:38.751088 16169 ts_tablet_manager.cc:1403] T 52ce1901589347d392b01d00179b4d68 P 613405f4f5894c14a690f9cb7dca93de: Time spent bootstrapping tablet: real 0.005s	user 0.004s	sys 0.000s
I20260812 06:37:38.753307 16169 raft_consensus.cc:359] T 52ce1901589347d392b01d00179b4d68 P 613405f4f5894c14a690f9cb7dca93de [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "613405f4f5894c14a690f9cb7dca93de" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 35913 } } peers { permanent_uuid: "d483e6d3aaf142f195a69b47f375a17f" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 41823 } } peers { permanent_uuid: "69b57269cd3e42c5bdc16a53a4e89240" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 37009 } }
I20260812 06:37:38.753552 16169 raft_consensus.cc:740] T 52ce1901589347d392b01d00179b4d68 P 613405f4f5894c14a690f9cb7dca93de [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 613405f4f5894c14a690f9cb7dca93de, State: Initialized, Role: FOLLOWER
I20260812 06:37:38.753571 16164 raft_consensus.cc:359] T 52ce1901589347d392b01d00179b4d68 P d483e6d3aaf142f195a69b47f375a17f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "613405f4f5894c14a690f9cb7dca93de" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 35913 } } peers { permanent_uuid: "d483e6d3aaf142f195a69b47f375a17f" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 41823 } } peers { permanent_uuid: "69b57269cd3e42c5bdc16a53a4e89240" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 37009 } }
I20260812 06:37:38.753572 16168 raft_consensus.cc:359] T 52ce1901589347d392b01d00179b4d68 P 69b57269cd3e42c5bdc16a53a4e89240 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "613405f4f5894c14a690f9cb7dca93de" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 35913 } } peers { permanent_uuid: "d483e6d3aaf142f195a69b47f375a17f" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 41823 } } peers { permanent_uuid: "69b57269cd3e42c5bdc16a53a4e89240" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 37009 } }
I20260812 06:37:38.753746 16168 raft_consensus.cc:740] T 52ce1901589347d392b01d00179b4d68 P 69b57269cd3e42c5bdc16a53a4e89240 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 69b57269cd3e42c5bdc16a53a4e89240, State: Initialized, Role: FOLLOWER
I20260812 06:37:38.753753 16164 raft_consensus.cc:740] T 52ce1901589347d392b01d00179b4d68 P d483e6d3aaf142f195a69b47f375a17f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d483e6d3aaf142f195a69b47f375a17f, State: Initialized, Role: FOLLOWER
I20260812 06:37:38.753849 16169 consensus_queue.cc:260] T 52ce1901589347d392b01d00179b4d68 P 613405f4f5894c14a690f9cb7dca93de [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "613405f4f5894c14a690f9cb7dca93de" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 35913 } } peers { permanent_uuid: "d483e6d3aaf142f195a69b47f375a17f" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 41823 } } peers { permanent_uuid: "69b57269cd3e42c5bdc16a53a4e89240" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 37009 } }
I20260812 06:37:38.754030 16168 consensus_queue.cc:260] T 52ce1901589347d392b01d00179b4d68 P 69b57269cd3e42c5bdc16a53a4e89240 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "613405f4f5894c14a690f9cb7dca93de" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 35913 } } peers { permanent_uuid: "d483e6d3aaf142f195a69b47f375a17f" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 41823 } } peers { permanent_uuid: "69b57269cd3e42c5bdc16a53a4e89240" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 37009 } }
I20260812 06:37:38.754029 16164 consensus_queue.cc:260] T 52ce1901589347d392b01d00179b4d68 P d483e6d3aaf142f195a69b47f375a17f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "613405f4f5894c14a690f9cb7dca93de" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 35913 } } peers { permanent_uuid: "d483e6d3aaf142f195a69b47f375a17f" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 41823 } } peers { permanent_uuid: "69b57269cd3e42c5bdc16a53a4e89240" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 37009 } }
I20260812 06:37:38.754549 15902 heartbeater.cc:499] Master 127.4.177.62:43119 was elected leader, sending a full tablet report...
I20260812 06:37:38.754838 16164 ts_tablet_manager.cc:1434] T 52ce1901589347d392b01d00179b4d68 P d483e6d3aaf142f195a69b47f375a17f: Time spent starting tablet: real 0.004s	user 0.002s	sys 0.002s
I20260812 06:37:38.754838 16168 ts_tablet_manager.cc:1434] T 52ce1901589347d392b01d00179b4d68 P 69b57269cd3e42c5bdc16a53a4e89240: Time spent starting tablet: real 0.004s	user 0.001s	sys 0.003s
I20260812 06:37:38.755168 16135 heartbeater.cc:499] Master 127.4.177.62:43119 was elected leader, sending a full tablet report...
I20260812 06:37:38.755357 15682 heartbeater.cc:499] Master 127.4.177.62:43119 was elected leader, sending a full tablet report...
I20260812 06:37:38.755748 16169 ts_tablet_manager.cc:1434] T 52ce1901589347d392b01d00179b4d68 P 613405f4f5894c14a690f9cb7dca93de: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260812 06:37:38.763789  4804 external_mini_cluster.cc:955] 3 TS(s) registered with all masters
I20260812 06:37:38.765065  4804 ts_itest-base.cc:246] Waiting for 1 tablets on tserver 69b57269cd3e42c5bdc16a53a4e89240 to finish bootstrapping
I20260812 06:37:38.770762  4804 ts_itest-base.cc:246] Waiting for 1 tablets on tserver 613405f4f5894c14a690f9cb7dca93de to finish bootstrapping
W20260812 06:37:38.775561 15903 tablet.cc:2406] T 52ce1901589347d392b01d00179b4d68 P 613405f4f5894c14a690f9cb7dca93de: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 06:37:38.776120  4804 ts_itest-base.cc:246] Waiting for 1 tablets on tserver d483e6d3aaf142f195a69b47f375a17f to finish bootstrapping
I20260812 06:37:38.784729 15822 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "52ce1901589347d392b01d00179b4d68"
dest_uuid: "613405f4f5894c14a690f9cb7dca93de"
 from {username='slave'} at 127.0.0.1:44874
I20260812 06:37:38.784955 15822 raft_consensus.cc:493] T 52ce1901589347d392b01d00179b4d68 P 613405f4f5894c14a690f9cb7dca93de [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260812 06:37:38.785092 15822 raft_consensus.cc:3060] T 52ce1901589347d392b01d00179b4d68 P 613405f4f5894c14a690f9cb7dca93de [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:37:38.786002 15822 raft_consensus.cc:515] T 52ce1901589347d392b01d00179b4d68 P 613405f4f5894c14a690f9cb7dca93de [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "613405f4f5894c14a690f9cb7dca93de" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 35913 } } peers { permanent_uuid: "d483e6d3aaf142f195a69b47f375a17f" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 41823 } } peers { permanent_uuid: "69b57269cd3e42c5bdc16a53a4e89240" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 37009 } }
I20260812 06:37:38.787134 15822 leader_election.cc:290] T 52ce1901589347d392b01d00179b4d68 P 613405f4f5894c14a690f9cb7dca93de [CANDIDATE]: Term 1 election: Requested vote from peers d483e6d3aaf142f195a69b47f375a17f (127.4.177.3:41823), 69b57269cd3e42c5bdc16a53a4e89240 (127.4.177.1:37009)
I20260812 06:37:38.791085 16051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52ce1901589347d392b01d00179b4d68" candidate_uuid: "613405f4f5894c14a690f9cb7dca93de" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "d483e6d3aaf142f195a69b47f375a17f"
I20260812 06:37:38.791338 16051 raft_consensus.cc:3060] T 52ce1901589347d392b01d00179b4d68 P d483e6d3aaf142f195a69b47f375a17f [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:37:38.791576 15600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52ce1901589347d392b01d00179b4d68" candidate_uuid: "613405f4f5894c14a690f9cb7dca93de" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "69b57269cd3e42c5bdc16a53a4e89240"
I20260812 06:37:38.791778 15600 raft_consensus.cc:3060] T 52ce1901589347d392b01d00179b4d68 P 69b57269cd3e42c5bdc16a53a4e89240 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:37:38.792913 15600 raft_consensus.cc:2468] T 52ce1901589347d392b01d00179b4d68 P 69b57269cd3e42c5bdc16a53a4e89240 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 613405f4f5894c14a690f9cb7dca93de in term 1.
I20260812 06:37:38.792912 16051 raft_consensus.cc:2468] T 52ce1901589347d392b01d00179b4d68 P d483e6d3aaf142f195a69b47f375a17f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 613405f4f5894c14a690f9cb7dca93de in term 1.
I20260812 06:37:38.793398 15714 leader_election.cc:304] T 52ce1901589347d392b01d00179b4d68 P 613405f4f5894c14a690f9cb7dca93de [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 613405f4f5894c14a690f9cb7dca93de, 69b57269cd3e42c5bdc16a53a4e89240; no voters: 
I20260812 06:37:38.793706 16175 raft_consensus.cc:2804] T 52ce1901589347d392b01d00179b4d68 P 613405f4f5894c14a690f9cb7dca93de [term 1 FOLLOWER]: Leader election won for term 1
I20260812 06:37:38.793926 16175 raft_consensus.cc:697] T 52ce1901589347d392b01d00179b4d68 P 613405f4f5894c14a690f9cb7dca93de [term 1 LEADER]: Becoming Leader. State: Replica: 613405f4f5894c14a690f9cb7dca93de, State: Running, Role: LEADER
I20260812 06:37:38.794261 16175 consensus_queue.cc:237] T 52ce1901589347d392b01d00179b4d68 P 613405f4f5894c14a690f9cb7dca93de [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "613405f4f5894c14a690f9cb7dca93de" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 35913 } } peers { permanent_uuid: "d483e6d3aaf142f195a69b47f375a17f" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 41823 } } peers { permanent_uuid: "69b57269cd3e42c5bdc16a53a4e89240" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 37009 } }
I20260812 06:37:38.797924 15376 catalog_manager.cc:5719] T 52ce1901589347d392b01d00179b4d68 P 613405f4f5894c14a690f9cb7dca93de reported cstate change: term changed from 0 to 1, leader changed from <none> to 613405f4f5894c14a690f9cb7dca93de (127.4.177.2). New cstate: current_term: 1 leader_uuid: "613405f4f5894c14a690f9cb7dca93de" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "613405f4f5894c14a690f9cb7dca93de" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 35913 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d483e6d3aaf142f195a69b47f375a17f" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 41823 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "69b57269cd3e42c5bdc16a53a4e89240" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 37009 } health_report { overall_health: UNKNOWN } } }
W20260812 06:37:38.847429 15684 tablet.cc:2406] T 52ce1901589347d392b01d00179b4d68 P 69b57269cd3e42c5bdc16a53a4e89240: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 06:37:38.874317 16175 consensus_queue.cc:1048] T 52ce1901589347d392b01d00179b4d68 P 613405f4f5894c14a690f9cb7dca93de [LEADER]: Connected to new peer: Peer: permanent_uuid: "d483e6d3aaf142f195a69b47f375a17f" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 41823 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 06:37:38.880267 16186 consensus_queue.cc:1048] T 52ce1901589347d392b01d00179b4d68 P 613405f4f5894c14a690f9cb7dca93de [LEADER]: Connected to new peer: Peer: permanent_uuid: "69b57269cd3e42c5bdc16a53a4e89240" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 37009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 06:37:38.892706  4804 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskCnZu03/build/debug/bin/kudu with pid 15473
I20260812 06:37:38.915571  4804 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskCnZu03/build/debug/bin/kudu with pid 15687
I20260812 06:37:39.435611  4804 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskCnZu03/build/debug/bin/kudu with pid 15905
I20260812 06:37:39.450664  4804 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskCnZu03/build/debug/bin/kudu with pid 15332
2026-08-12T06:37:39Z chronyd exiting
[       OK ] RaftConsensusITest.TestElectionMetricsFailureDetectionDisabled (1537 ms)
[----------] 6 tests from RaftConsensusITest (90918 ms total)

[----------] 1 test from EncryptionEnabled/RaftConsensusParamEncryptionITest
[ RUN      ] EncryptionEnabled/RaftConsensusParamEncryptionITest.TestFollowerFallsBehindLeaderGC/0
I20260812 06:37:39.477849  4804 ts_itest-base.cc:115] Starting cluster with:
I20260812 06:37:39.477916  4804 ts_itest-base.cc:116] --------------
I20260812 06:37:39.477936  4804 ts_itest-base.cc:117] 3 tablet servers
I20260812 06:37:39.477953  4804 ts_itest-base.cc:118] 3 replicas per TS
I20260812 06:37:39.477980  4804 ts_itest-base.cc:119] --------------
2026-08-12T06:37:39Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-08-12T06:37:39Z Disabled control of system clock
I20260812 06:37:39.489570  4804 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskCnZu03/build/debug/bin/kudu
/tmp/dist-test-taskCnZu03/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.EncryptionEnabled_RaftConsensusParamEncryptionITest.TestFollowerFallsBehindLeaderGC_0.1786516568546459-4804-0/raft_consensus-itest-cluster/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.EncryptionEnabled_RaftConsensusParamEncryptionITest.TestFollowerFallsBehindLeaderGC_0.1786516568546459-4804-0/raft_consensus-itest-cluster/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.EncryptionEnabled_RaftConsensusParamEncryptionITest.TestFollowerFallsBehindLeaderGC_0.1786516568546459-4804-0/raft_consensus-itest-cluster/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.EncryptionEnabled_RaftConsensusParamEncryptionITest.TestFollowerFallsBehindLeaderGC_0.1786516568546459-4804-0/raft_consensus-itest-cluster/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.177.62:41039
--webserver_interface=127.4.177.62
--webserver_port=0
--builtin_ntp_servers=127.4.177.20:33149
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.177.62:41039 with env {}
W20260812 06:37:39.593633 16212 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:37:39.593890 16212 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:37:39.593989 16212 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 06:37:39.597338 16212 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 06:37:39.597430 16212 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:37:39.597468 16212 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 06:37:39.597504 16212 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
I20260812 06:37:39.601368 16212 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.EncryptionEnabled_RaftConsensusParamEncryptionITest.TestFollowerFallsBehindLeaderGC_0.1786516568546459-4804-0/raft_consensus-itest-cluster/master-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.177.20:33149
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.EncryptionEnabled_RaftConsensusParamEncryptionITest.TestFollowerFallsBehindLeaderGC_0.1786516568546459-4804-0/raft_consensus-itest-cluster/master-0/data
--fs_wal_dir=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.EncryptionEnabled_RaftConsensusParamEncryptionITest.TestFollowerFallsBehindLeaderGC_0.1786516568546459-4804-0/raft_consensus-itest-cluster/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.177.62:41039
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.177.62:41039
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.EncryptionEnabled_RaftConsensusParamEncryptionITest.TestFollowerFallsBehindLeaderGC_0.1786516568546459-4804-0/raft_consensus-itest-cluster/master-0/data/info.pb
--webserver_interface=127.4.177.62
--webserver_port=0
--never_fsync=true
--heap_profile_path=/tmp/kudu.16212
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false

Master server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 06:37:39.602535 16212 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:37:39.603619 16212 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 06:37:39.609228 16227 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:37:39.609241 16224 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:37:39.609437 16212 server_base.cc:1061] running on GCE node
W20260812 06:37:39.609218 16222 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:37:39.610105 16212 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:37:39.611014 16212 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:37:39.612195 16212 hybrid_clock.cc:648] HybridClock initialized: now 1786516659612185 us; error 33 us; skew 500 ppm
I20260812 06:37:39.613850 16212 webserver.cc:533] Webserver started at http://127.4.177.62:37113/ using document root <none> and password file <none>
I20260812 06:37:39.614506 16212 fs_manager.cc:362] Metadata directory not provided
I20260812 06:37:39.614593 16212 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:37:39.614813 16212 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:37:39.616297 16212 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.EncryptionEnabled_RaftConsensusParamEncryptionITest.TestFollowerFallsBehindLeaderGC_0.1786516568546459-4804-0/raft_consensus-itest-cluster/master-0/data/instance:
uuid: "bcdd0f32878d46d4a2285edc634460de"
format_stamp: "Formatted at 2026-08-12 06:37:39 on dist-test-slave-jztv"
I20260812 06:37:39.616787 16212 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.EncryptionEnabled_RaftConsensusParamEncryptionITest.TestFollowerFallsBehindLeaderGC_0.1786516568546459-4804-0/raft_consensus-itest-cluster/master-0/wal/instance:
uuid: "bcdd0f32878d46d4a2285edc634460de"
format_stamp: "Formatted at 2026-08-12 06:37:39 on dist-test-slave-jztv"
I20260812 06:37:39.620031 16212 fs_manager.cc:696] Time spent creating directory manager: real 0.003s	user 0.005s	sys 0.000s
I20260812 06:37:39.622103 16238 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 06:37:39.623076 16212 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260812 06:37:39.623209 16212 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.EncryptionEnabled_RaftConsensusParamEncryptionITest.TestFollowerFallsBehindLeaderGC_0.1786516568546459-4804-0/raft_consensus-itest-cluster/master-0/data,/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.EncryptionEnabled_RaftConsensusParamEncryptionITest.TestFollowerFallsBehindLeaderGC_0.1786516568546459-4804-0/raft_consensus-itest-cluster/master-0/wal
uuid: "bcdd0f32878d46d4a2285edc634460de"
format_stamp: "Formatted at 2026-08-12 06:37:39 on dist-test-slave-jztv"
I20260812 06:37:39.623310 16212 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.EncryptionEnabled_RaftConsensusParamEncryptionITest.TestFollowerFallsBehindLeaderGC_0.1786516568546459-4804-0/raft_consensus-itest-cluster/master-0/wal
metadata directory: /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.EncryptionEnabled_RaftConsensusParamEncryptionITest.TestFollowerFallsBehindLeaderGC_0.1786516568546459-4804-0/raft_consensus-itest-cluster/master-0/wal
1 data directories: /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.EncryptionEnabled_RaftConsensusParamEncryptionITest.TestFollowerFallsBehindLeaderGC_0.1786516568546459-4804-0/raft_consensus-itest-cluster/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:37:39.634918 16212 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 06:37:39.635602 16212 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:37:39.635799 16212 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:37:39.643584 16212 rpc_server.cc:307] RPC server started. Bound to: 127.4.177.62:41039
I20260812 06:37:39.643579 16324 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.4.177.62:41039 every 8 connection(s)
I20260812 06:37:39.644655 16212 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.EncryptionEnabled_RaftConsensusParamEncryptionITest.TestFollowerFallsBehindLeaderGC_0.1786516568546459-4804-0/raft_consensus-itest-cluster/master-0/data/info.pb
I20260812 06:37:39.645628  4804 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskCnZu03/build/debug/bin/kudu as pid 16212
I20260812 06:37:39.645756  4804 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.EncryptionEnabled_RaftConsensusParamEncryptionITest.TestFollowerFallsBehindLeaderGC_0.1786516568546459-4804-0/raft_consensus-itest-cluster/master-0/wal/instance
I20260812 06:37:39.648355 16326 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:37:39.653086 16326 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bcdd0f32878d46d4a2285edc634460de: Bootstrap starting.
I20260812 06:37:39.655460 16326 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bcdd0f32878d46d4a2285edc634460de: Neither blocks nor log segments found. Creating new log.
I20260812 06:37:39.656278 16326 log.cc:826] T 00000000000000000000000000000000 P bcdd0f32878d46d4a2285edc634460de: Log is configured to *not* fsync() on all Append() calls
I20260812 06:37:39.657918 16326 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bcdd0f32878d46d4a2285edc634460de: No bootstrap required, opened a new log
I20260812 06:37:39.660778 16326 raft_consensus.cc:359] T 00000000000000000000000000000000 P bcdd0f32878d46d4a2285edc634460de [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bcdd0f32878d46d4a2285edc634460de" member_type: VOTER last_known_addr { host: "127.4.177.62" port: 41039 } }
I20260812 06:37:39.660955 16326 raft_consensus.cc:385] T 00000000000000000000000000000000 P bcdd0f32878d46d4a2285edc634460de [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:37:39.660989 16326 raft_consensus.cc:740] T 00000000000000000000000000000000 P bcdd0f32878d46d4a2285edc634460de [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bcdd0f32878d46d4a2285edc634460de, State: Initialized, Role: FOLLOWER
I20260812 06:37:39.661378 16326 consensus_queue.cc:260] T 00000000000000000000000000000000 P bcdd0f32878d46d4a2285edc634460de [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bcdd0f32878d46d4a2285edc634460de" member_type: VOTER last_known_addr { host: "127.4.177.62" port: 41039 } }
I20260812 06:37:39.661499 16326 raft_consensus.cc:399] T 00000000000000000000000000000000 P bcdd0f32878d46d4a2285edc634460de [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 06:37:39.661549 16326 raft_consensus.cc:493] T 00000000000000000000000000000000 P bcdd0f32878d46d4a2285edc634460de [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 06:37:39.661628 16326 raft_consensus.cc:3060] T 00000000000000000000000000000000 P bcdd0f32878d46d4a2285edc634460de [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:37:39.662461 16326 raft_consensus.cc:515] T 00000000000000000000000000000000 P bcdd0f32878d46d4a2285edc634460de [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bcdd0f32878d46d4a2285edc634460de" member_type: VOTER last_known_addr { host: "127.4.177.62" port: 41039 } }
I20260812 06:37:39.662895 16326 leader_election.cc:304] T 00000000000000000000000000000000 P bcdd0f32878d46d4a2285edc634460de [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bcdd0f32878d46d4a2285edc634460de; no voters: 
I20260812 06:37:39.663249 16326 leader_election.cc:290] T 00000000000000000000000000000000 P bcdd0f32878d46d4a2285edc634460de [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260812 06:37:39.663334 16332 raft_consensus.cc:2804] T 00000000000000000000000000000000 P bcdd0f32878d46d4a2285edc634460de [term 1 FOLLOWER]: Leader election won for term 1
I20260812 06:37:39.663620 16332 raft_consensus.cc:697] T 00000000000000000000000000000000 P bcdd0f32878d46d4a2285edc634460de [term 1 LEADER]: Becoming Leader. State: Replica: bcdd0f32878d46d4a2285edc634460de, State: Running, Role: LEADER
I20260812 06:37:39.663924 16332 consensus_queue.cc:237] T 00000000000000000000000000000000 P bcdd0f32878d46d4a2285edc634460de [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bcdd0f32878d46d4a2285edc634460de" member_type: VOTER last_known_addr { host: "127.4.177.62" port: 41039 } }
I20260812 06:37:39.664474 16326 sys_catalog.cc:565] T 00000000000000000000000000000000 P bcdd0f32878d46d4a2285edc634460de [sys.catalog]: configured and running, proceeding with master startup.
I20260812 06:37:39.665587 16333 sys_catalog.cc:455] T 00000000000000000000000000000000 P bcdd0f32878d46d4a2285edc634460de [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bcdd0f32878d46d4a2285edc634460de" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bcdd0f32878d46d4a2285edc634460de" member_type: VOTER last_known_addr { host: "127.4.177.62" port: 41039 } } }
I20260812 06:37:39.665740 16333 sys_catalog.cc:458] T 00000000000000000000000000000000 P bcdd0f32878d46d4a2285edc634460de [sys.catalog]: This master's current role is: LEADER
I20260812 06:37:39.666087 16348 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260812 06:37:39.666127 16338 sys_catalog.cc:455] T 00000000000000000000000000000000 P bcdd0f32878d46d4a2285edc634460de [sys.catalog]: SysCatalogTable state changed. Reason: New leader bcdd0f32878d46d4a2285edc634460de. Latest consensus state: current_term: 1 leader_uuid: "bcdd0f32878d46d4a2285edc634460de" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bcdd0f32878d46d4a2285edc634460de" member_type: VOTER last_known_addr { host: "127.4.177.62" port: 41039 } } }
I20260812 06:37:39.666257 16338 sys_catalog.cc:458] T 00000000000000000000000000000000 P bcdd0f32878d46d4a2285edc634460de [sys.catalog]: This master's current role is: LEADER
I20260812 06:37:39.669166 16348 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260812 06:37:39.674770 16348 catalog_manager.cc:1383] Generated new cluster ID: 8e5a955ade8d4842aad1898f30d414e6
I20260812 06:37:39.674849 16348 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260812 06:37:39.691579 16348 catalog_manager.cc:1406] Generated new certificate authority record
I20260812 06:37:39.692461 16348 catalog_manager.cc:1540] Loading token signing keys...
I20260812 06:37:39.699683 16348 catalog_manager.cc:6092] T 00000000000000000000000000000000 P bcdd0f32878d46d4a2285edc634460de: Generated new TSK 0
I20260812 06:37:39.700537 16348 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260812 06:37:39.704164  4804 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskCnZu03/build/debug/bin/kudu
/tmp/dist-test-taskCnZu03/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.EncryptionEnabled_RaftConsensusParamEncryptionITest.TestFollowerFallsBehindLeaderGC_0.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.EncryptionEnabled_RaftConsensusParamEncryptionITest.TestFollowerFallsBehindLeaderGC_0.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.EncryptionEnabled_RaftConsensusParamEncryptionITest.TestFollowerFallsBehindLeaderGC_0.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.EncryptionEnabled_RaftConsensusParamEncryptionITest.TestFollowerFallsBehindLeaderGC_0.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.177.1:0
--local_ip_for_outbound_sockets=127.4.177.1
--webserver_interface=127.4.177.1
--webserver_port=0
--tserver_master_addrs=127.4.177.62:41039
--builtin_ntp_servers=127.4.177.20:33149
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--evict_failed_followers=false
--log_compression_codec=no_compression
--log_cache_size_limit_mb=1
--log_segment_size_mb=1
--log_async_preallocate_segments=false
--log_min_segments_to_retain=1
--log_max_segments_to_retain=3
--maintenance_manager_polling_interval_ms=100
--log_target_replay_size_mb=1
--max_cell_size_bytes=1000000 with env {}
W20260812 06:37:39.810654 16365 flags.cc:432] Enabled unsafe flag: --max_cell_size_bytes=1000000
W20260812 06:37:39.810880 16365 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:37:39.810901 16365 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:37:39.810998 16365 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 06:37:39.814366 16365 flags.cc:432] Enabled experimental flag: --log_compression_codec=no_compression
W20260812 06:37:39.814422 16365 flags.cc:432] Enabled experimental flag: --log_max_segments_to_retain=3
W20260812 06:37:39.814460 16365 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:37:39.814575 16365 flags.cc:432] Enabled experimental flag: --log_target_replay_size_mb=1
W20260812 06:37:39.814610 16365 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.177.1
I20260812 06:37:39.818526 16365 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.EncryptionEnabled_RaftConsensusParamEncryptionITest.TestFollowerFallsBehindLeaderGC_0.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.177.20:33149
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--max_cell_size_bytes=1000000
--log_compression_codec=no_compression
--log_max_segments_to_retain=3
--log_cache_size_limit_mb=1
--log_async_preallocate_segments=false
--log_segment_size_mb=1
--evict_failed_followers=false
--fs_data_dirs=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.EncryptionEnabled_RaftConsensusParamEncryptionITest.TestFollowerFallsBehindLeaderGC_0.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.EncryptionEnabled_RaftConsensusParamEncryptionITest.TestFollowerFallsBehindLeaderGC_0.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.177.1:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.EncryptionEnabled_RaftConsensusParamEncryptionITest.TestFollowerFallsBehindLeaderGC_0.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-0/data/info.pb
--webserver_interface=127.4.177.1
--webserver_port=0
--tserver_master_addrs=127.4.177.62:41039
--never_fsync=true
--heap_profile_path=/tmp/kudu.16365
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--log_target_replay_size_mb=1
--maintenance_manager_polling_interval_ms=100
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.177.1

Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 06:37:39.819626 16365 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:37:39.820739 16365 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 06:37:39.827019 16373 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:37:39.827039 16377 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:37:39.827203 16365 server_base.cc:1061] running on GCE node
W20260812 06:37:39.827359 16374 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:37:39.827839 16365 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:37:39.828402 16365 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:37:39.829577 16365 hybrid_clock.cc:648] HybridClock initialized: now 1786516659829560 us; error 38 us; skew 500 ppm
I20260812 06:37:39.831391 16365 webserver.cc:533] Webserver started at http://127.4.177.1:39495/ using document root <none> and password file <none>
I20260812 06:37:39.831949 16365 fs_manager.cc:362] Metadata directory not provided
I20260812 06:37:39.832037 16365 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:37:39.832258 16365 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:37:39.833792 16365 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.EncryptionEnabled_RaftConsensusParamEncryptionITest.TestFollowerFallsBehindLeaderGC_0.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-0/data/instance:
uuid: "68b666b388e54d3a93ae86c6098c7b44"
format_stamp: "Formatted at 2026-08-12 06:37:39 on dist-test-slave-jztv"
I20260812 06:37:39.834344 16365 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.EncryptionEnabled_RaftConsensusParamEncryptionITest.TestFollowerFallsBehindLeaderGC_0.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-0/wal/instance:
uuid: "68b666b388e54d3a93ae86c6098c7b44"
format_stamp: "Formatted at 2026-08-12 06:37:39 on dist-test-slave-jztv"
I20260812 06:37:39.837647 16365 fs_manager.cc:696] Time spent creating directory manager: real 0.003s	user 0.000s	sys 0.003s
I20260812 06:37:39.839995 16387 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 06:37:39.841032 16365 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.001s	sys 0.000s
I20260812 06:37:39.841159 16365 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.EncryptionEnabled_RaftConsensusParamEncryptionITest.TestFollowerFallsBehindLeaderGC_0.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-0/data,/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.EncryptionEnabled_RaftConsensusParamEncryptionITest.TestFollowerFallsBehindLeaderGC_0.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-0/wal
uuid: "68b666b388e54d3a93ae86c6098c7b44"
format_stamp: "Formatted at 2026-08-12 06:37:39 on dist-test-slave-jztv"
I20260812 06:37:39.841262 16365 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.EncryptionEnabled_RaftConsensusParamEncryptionITest.TestFollowerFallsBehindLeaderGC_0.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-0/wal
metadata directory: /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.EncryptionEnabled_RaftConsensusParamEncryptionITest.TestFollowerFallsBehindLeaderGC_0.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-0/wal
1 data directories: /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.EncryptionEnabled_RaftConsensusParamEncryptionITest.TestFollowerFallsBehindLeaderGC_0.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:37:39.860918 16365 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 06:37:39.861521 16365 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:37:39.861704 16365 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:37:39.862303 16365 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260812 06:37:39.863276 16365 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:37:39.863344 16365 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260812 06:37:39.863404 16365 ts_tablet_manager.cc:616] Registered 0 tablets
I20260812 06:37:39.863446 16365 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260812 06:37:39.872887 16365 rpc_server.cc:307] RPC server started. Bound to: 127.4.177.1:40995
I20260812 06:37:39.872927 16583 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.4.177.1:40995 every 8 connection(s)
I20260812 06:37:39.873785 16365 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.EncryptionEnabled_RaftConsensusParamEncryptionITest.TestFollowerFallsBehindLeaderGC_0.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-0/data/info.pb
I20260812 06:37:39.882488 16584 heartbeater.cc:344] Connected to a master server at 127.4.177.62:41039
I20260812 06:37:39.882730 16584 heartbeater.cc:461] Registering TS with master...
I20260812 06:37:39.882756  4804 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskCnZu03/build/debug/bin/kudu as pid 16365
I20260812 06:37:39.882882  4804 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.EncryptionEnabled_RaftConsensusParamEncryptionITest.TestFollowerFallsBehindLeaderGC_0.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-0/wal/instance
I20260812 06:37:39.883260 16584 heartbeater.cc:507] Master 127.4.177.62:41039 requested a full tablet report, sending...
I20260812 06:37:39.884919 16268 ts_manager.cc:194] Registered new tserver with Master: 68b666b388e54d3a93ae86c6098c7b44 (127.4.177.1:40995)
I20260812 06:37:39.885260  4804 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskCnZu03/build/debug/bin/kudu
/tmp/dist-test-taskCnZu03/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.EncryptionEnabled_RaftConsensusParamEncryptionITest.TestFollowerFallsBehindLeaderGC_0.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.EncryptionEnabled_RaftConsensusParamEncryptionITest.TestFollowerFallsBehindLeaderGC_0.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.EncryptionEnabled_RaftConsensusParamEncryptionITest.TestFollowerFallsBehindLeaderGC_0.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.EncryptionEnabled_RaftConsensusParamEncryptionITest.TestFollowerFallsBehindLeaderGC_0.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.177.2:0
--local_ip_for_outbound_sockets=127.4.177.2
--webserver_interface=127.4.177.2
--webserver_port=0
--tserver_master_addrs=127.4.177.62:41039
--builtin_ntp_servers=127.4.177.20:33149
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--evict_failed_followers=false
--log_compression_codec=no_compression
--log_cache_size_limit_mb=1
--log_segment_size_mb=1
--log_async_preallocate_segments=false
--log_min_segments_to_retain=1
--log_max_segments_to_retain=3
--maintenance_manager_polling_interval_ms=100
--log_target_replay_size_mb=1
--max_cell_size_bytes=1000000 with env {}
I20260812 06:37:39.886289 16268 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.4.177.1:57565
W20260812 06:37:39.990401 16588 flags.cc:432] Enabled unsafe flag: --max_cell_size_bytes=1000000
W20260812 06:37:39.990626 16588 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:37:39.990648 16588 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:37:39.990687 16588 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 06:37:39.993930 16588 flags.cc:432] Enabled experimental flag: --log_compression_codec=no_compression
W20260812 06:37:39.993988 16588 flags.cc:432] Enabled experimental flag: --log_max_segments_to_retain=3
W20260812 06:37:39.994024 16588 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:37:39.994083 16588 flags.cc:432] Enabled experimental flag: --log_target_replay_size_mb=1
W20260812 06:37:39.994104 16588 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.177.2
I20260812 06:37:39.997830 16588 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.EncryptionEnabled_RaftConsensusParamEncryptionITest.TestFollowerFallsBehindLeaderGC_0.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-1/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.177.20:33149
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--max_cell_size_bytes=1000000
--log_compression_codec=no_compression
--log_max_segments_to_retain=3
--log_cache_size_limit_mb=1
--log_async_preallocate_segments=false
--log_segment_size_mb=1
--evict_failed_followers=false
--fs_data_dirs=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.EncryptionEnabled_RaftConsensusParamEncryptionITest.TestFollowerFallsBehindLeaderGC_0.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.EncryptionEnabled_RaftConsensusParamEncryptionITest.TestFollowerFallsBehindLeaderGC_0.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.177.2:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.EncryptionEnabled_RaftConsensusParamEncryptionITest.TestFollowerFallsBehindLeaderGC_0.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-1/data/info.pb
--webserver_interface=127.4.177.2
--webserver_port=0
--tserver_master_addrs=127.4.177.62:41039
--never_fsync=true
--heap_profile_path=/tmp/kudu.16588
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--log_target_replay_size_mb=1
--maintenance_manager_polling_interval_ms=100
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.177.2

Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 06:37:39.998823 16588 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:37:39.999881 16588 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 06:37:40.006264 16600 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:37:40.006338 16599 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:37:40.006379 16588 server_base.cc:1061] running on GCE node
W20260812 06:37:40.006276 16603 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:37:40.006927 16588 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:37:40.007483 16588 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:37:40.008672 16588 hybrid_clock.cc:648] HybridClock initialized: now 1786516660008658 us; error 31 us; skew 500 ppm
I20260812 06:37:40.010532 16588 webserver.cc:533] Webserver started at http://127.4.177.2:35145/ using document root <none> and password file <none>
I20260812 06:37:40.011111 16588 fs_manager.cc:362] Metadata directory not provided
I20260812 06:37:40.011199 16588 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:37:40.011423 16588 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:37:40.013047 16588 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.EncryptionEnabled_RaftConsensusParamEncryptionITest.TestFollowerFallsBehindLeaderGC_0.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-1/data/instance:
uuid: "d42658cb732d4733b992c5e7a7540576"
format_stamp: "Formatted at 2026-08-12 06:37:40 on dist-test-slave-jztv"
I20260812 06:37:40.013540 16588 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.EncryptionEnabled_RaftConsensusParamEncryptionITest.TestFollowerFallsBehindLeaderGC_0.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-1/wal/instance:
uuid: "d42658cb732d4733b992c5e7a7540576"
format_stamp: "Formatted at 2026-08-12 06:37:40 on dist-test-slave-jztv"
I20260812 06:37:40.016846 16588 fs_manager.cc:696] Time spent creating directory manager: real 0.003s	user 0.006s	sys 0.000s
I20260812 06:37:40.019176 16613 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 06:37:40.020174 16588 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260812 06:37:40.020304 16588 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.EncryptionEnabled_RaftConsensusParamEncryptionITest.TestFollowerFallsBehindLeaderGC_0.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-1/data,/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.EncryptionEnabled_RaftConsensusParamEncryptionITest.TestFollowerFallsBehindLeaderGC_0.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-1/wal
uuid: "d42658cb732d4733b992c5e7a7540576"
format_stamp: "Formatted at 2026-08-12 06:37:40 on dist-test-slave-jztv"
I20260812 06:37:40.020404 16588 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.EncryptionEnabled_RaftConsensusParamEncryptionITest.TestFollowerFallsBehindLeaderGC_0.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-1/wal
metadata directory: /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.EncryptionEnabled_RaftConsensusParamEncryptionITest.TestFollowerFallsBehindLeaderGC_0.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-1/wal
1 data directories: /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.EncryptionEnabled_RaftConsensusParamEncryptionITest.TestFollowerFallsBehindLeaderGC_0.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:37:40.051771 16588 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 06:37:40.052502 16588 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:37:40.052708 16588 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:37:40.053303 16588 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260812 06:37:40.054345 16588 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:37:40.054431 16588 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260812 06:37:40.054505 16588 ts_tablet_manager.cc:616] Registered 0 tablets
I20260812 06:37:40.054559 16588 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260812 06:37:40.063966 16588 rpc_server.cc:307] RPC server started. Bound to: 127.4.177.2:38765
I20260812 06:37:40.064002 16824 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.4.177.2:38765 every 8 connection(s)
I20260812 06:37:40.064936 16588 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.EncryptionEnabled_RaftConsensusParamEncryptionITest.TestFollowerFallsBehindLeaderGC_0.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-1/data/info.pb
I20260812 06:37:40.073788 16827 heartbeater.cc:344] Connected to a master server at 127.4.177.62:41039
I20260812 06:37:40.074059 16827 heartbeater.cc:461] Registering TS with master...
I20260812 06:37:40.074091  4804 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskCnZu03/build/debug/bin/kudu as pid 16588
I20260812 06:37:40.074254  4804 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.EncryptionEnabled_RaftConsensusParamEncryptionITest.TestFollowerFallsBehindLeaderGC_0.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-1/wal/instance
I20260812 06:37:40.074680 16827 heartbeater.cc:507] Master 127.4.177.62:41039 requested a full tablet report, sending...
I20260812 06:37:40.075831 16269 ts_manager.cc:194] Registered new tserver with Master: d42658cb732d4733b992c5e7a7540576 (127.4.177.2:38765)
I20260812 06:37:40.076489 16269 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.4.177.2:53115
I20260812 06:37:40.076972  4804 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskCnZu03/build/debug/bin/kudu
/tmp/dist-test-taskCnZu03/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.EncryptionEnabled_RaftConsensusParamEncryptionITest.TestFollowerFallsBehindLeaderGC_0.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.EncryptionEnabled_RaftConsensusParamEncryptionITest.TestFollowerFallsBehindLeaderGC_0.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.EncryptionEnabled_RaftConsensusParamEncryptionITest.TestFollowerFallsBehindLeaderGC_0.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.EncryptionEnabled_RaftConsensusParamEncryptionITest.TestFollowerFallsBehindLeaderGC_0.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.177.3:0
--local_ip_for_outbound_sockets=127.4.177.3
--webserver_interface=127.4.177.3
--webserver_port=0
--tserver_master_addrs=127.4.177.62:41039
--builtin_ntp_servers=127.4.177.20:33149
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--evict_failed_followers=false
--log_compression_codec=no_compression
--log_cache_size_limit_mb=1
--log_segment_size_mb=1
--log_async_preallocate_segments=false
--log_min_segments_to_retain=1
--log_max_segments_to_retain=3
--maintenance_manager_polling_interval_ms=100
--log_target_replay_size_mb=1
--max_cell_size_bytes=1000000 with env {}
W20260812 06:37:40.186827 16831 flags.cc:432] Enabled unsafe flag: --max_cell_size_bytes=1000000
W20260812 06:37:40.187079 16831 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 06:37:40.187103 16831 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 06:37:40.187140 16831 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 06:37:40.190402 16831 flags.cc:432] Enabled experimental flag: --log_compression_codec=no_compression
W20260812 06:37:40.190459 16831 flags.cc:432] Enabled experimental flag: --log_max_segments_to_retain=3
W20260812 06:37:40.190496 16831 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 06:37:40.190557 16831 flags.cc:432] Enabled experimental flag: --log_target_replay_size_mb=1
W20260812 06:37:40.190584 16831 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.177.3
I20260812 06:37:40.194408 16831 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.EncryptionEnabled_RaftConsensusParamEncryptionITest.TestFollowerFallsBehindLeaderGC_0.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-2/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.177.20:33149
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--max_cell_size_bytes=1000000
--log_compression_codec=no_compression
--log_max_segments_to_retain=3
--log_cache_size_limit_mb=1
--log_async_preallocate_segments=false
--log_segment_size_mb=1
--evict_failed_followers=false
--fs_data_dirs=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.EncryptionEnabled_RaftConsensusParamEncryptionITest.TestFollowerFallsBehindLeaderGC_0.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.EncryptionEnabled_RaftConsensusParamEncryptionITest.TestFollowerFallsBehindLeaderGC_0.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.177.3:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.EncryptionEnabled_RaftConsensusParamEncryptionITest.TestFollowerFallsBehindLeaderGC_0.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-2/data/info.pb
--webserver_interface=127.4.177.3
--webserver_port=0
--tserver_master_addrs=127.4.177.62:41039
--never_fsync=true
--heap_profile_path=/tmp/kudu.16831
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--log_target_replay_size_mb=1
--maintenance_manager_polling_interval_ms=100
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.177.3

Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 06:37:40.195474 16831 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 06:37:40.196532 16831 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 06:37:40.202644 16840 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:37:40.202623 16843 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 06:37:40.202754 16839 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 06:37:40.202991 16831 server_base.cc:1061] running on GCE node
I20260812 06:37:40.203399 16831 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 06:37:40.203984 16831 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 06:37:40.205165 16831 hybrid_clock.cc:648] HybridClock initialized: now 1786516660205145 us; error 29 us; skew 500 ppm
I20260812 06:37:40.207026 16831 webserver.cc:533] Webserver started at http://127.4.177.3:35503/ using document root <none> and password file <none>
I20260812 06:37:40.207633 16831 fs_manager.cc:362] Metadata directory not provided
I20260812 06:37:40.207695 16831 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 06:37:40.207937 16831 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 06:37:40.209504 16831 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.EncryptionEnabled_RaftConsensusParamEncryptionITest.TestFollowerFallsBehindLeaderGC_0.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-2/data/instance:
uuid: "d41d668ab84c421381071558fe019747"
format_stamp: "Formatted at 2026-08-12 06:37:40 on dist-test-slave-jztv"
I20260812 06:37:40.209991 16831 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.EncryptionEnabled_RaftConsensusParamEncryptionITest.TestFollowerFallsBehindLeaderGC_0.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-2/wal/instance:
uuid: "d41d668ab84c421381071558fe019747"
format_stamp: "Formatted at 2026-08-12 06:37:40 on dist-test-slave-jztv"
I20260812 06:37:40.213686 16831 fs_manager.cc:696] Time spent creating directory manager: real 0.003s	user 0.002s	sys 0.001s
I20260812 06:37:40.216198 16853 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 06:37:40.217227 16831 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260812 06:37:40.217352 16831 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.EncryptionEnabled_RaftConsensusParamEncryptionITest.TestFollowerFallsBehindLeaderGC_0.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-2/data,/tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.EncryptionEnabled_RaftConsensusParamEncryptionITest.TestFollowerFallsBehindLeaderGC_0.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-2/wal
uuid: "d41d668ab84c421381071558fe019747"
format_stamp: "Formatted at 2026-08-12 06:37:40 on dist-test-slave-jztv"
I20260812 06:37:40.217455 16831 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.EncryptionEnabled_RaftConsensusParamEncryptionITest.TestFollowerFallsBehindLeaderGC_0.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-2/wal
metadata directory: /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.EncryptionEnabled_RaftConsensusParamEncryptionITest.TestFollowerFallsBehindLeaderGC_0.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-2/wal
1 data directories: /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.EncryptionEnabled_RaftConsensusParamEncryptionITest.TestFollowerFallsBehindLeaderGC_0.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 06:37:40.226863 16831 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 06:37:40.227595 16831 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 06:37:40.227785 16831 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 06:37:40.228305 16831 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260812 06:37:40.229261 16831 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 06:37:40.229311 16831 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260812 06:37:40.229350 16831 ts_tablet_manager.cc:616] Registered 0 tablets
I20260812 06:37:40.229367 16831 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260812 06:37:40.238610 16831 rpc_server.cc:307] RPC server started. Bound to: 127.4.177.3:36879
I20260812 06:37:40.238631 17050 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.4.177.3:36879 every 8 connection(s)
I20260812 06:37:40.239629 16831 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.EncryptionEnabled_RaftConsensusParamEncryptionITest.TestFollowerFallsBehindLeaderGC_0.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-2/data/info.pb
I20260812 06:37:40.245949  4804 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskCnZu03/build/debug/bin/kudu as pid 16831
I20260812 06:37:40.246124  4804 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.EncryptionEnabled_RaftConsensusParamEncryptionITest.TestFollowerFallsBehindLeaderGC_0.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-2/wal/instance
I20260812 06:37:40.249089 17051 heartbeater.cc:344] Connected to a master server at 127.4.177.62:41039
I20260812 06:37:40.249351 17051 heartbeater.cc:461] Registering TS with master...
I20260812 06:37:40.249855 17051 heartbeater.cc:507] Master 127.4.177.62:41039 requested a full tablet report, sending...
I20260812 06:37:40.250928 16269 ts_manager.cc:194] Registered new tserver with Master: d41d668ab84c421381071558fe019747 (127.4.177.3:36879)
I20260812 06:37:40.251487 16269 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.4.177.3:56983
I20260812 06:37:40.260200  4804 external_mini_cluster.cc:955] 3 TS(s) registered with all masters
I20260812 06:37:40.268759 16269 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60844:
name: "TestTable"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
owner: "alice"
W20260812 06:37:40.271152 16269 catalog_manager.cc:7099] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table TestTable in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260812 06:37:40.289381 16931 tablet_service.cc:1511] Processing CreateTablet for tablet c0b0a910c55b4f928acd0a5e4b346673 (DEFAULT_TABLE table=TestTable [id=1d35b3f1c7c34c218d722c94878d0562]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 06:37:40.290462 16931 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c0b0a910c55b4f928acd0a5e4b346673. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:37:40.290762 16468 tablet_service.cc:1511] Processing CreateTablet for tablet c0b0a910c55b4f928acd0a5e4b346673 (DEFAULT_TABLE table=TestTable [id=1d35b3f1c7c34c218d722c94878d0562]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 06:37:40.291630 16468 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c0b0a910c55b4f928acd0a5e4b346673. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:37:40.291672 16703 tablet_service.cc:1511] Processing CreateTablet for tablet c0b0a910c55b4f928acd0a5e4b346673 (DEFAULT_TABLE table=TestTable [id=1d35b3f1c7c34c218d722c94878d0562]), partition=RANGE (key) PARTITION UNBOUNDED
I20260812 06:37:40.292309 16703 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c0b0a910c55b4f928acd0a5e4b346673. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:37:40.296409 17076 tablet_bootstrap.cc:492] T c0b0a910c55b4f928acd0a5e4b346673 P d41d668ab84c421381071558fe019747: Bootstrap starting.
I20260812 06:37:40.297812 17077 tablet_bootstrap.cc:492] T c0b0a910c55b4f928acd0a5e4b346673 P d42658cb732d4733b992c5e7a7540576: Bootstrap starting.
I20260812 06:37:40.298547 17079 tablet_bootstrap.cc:492] T c0b0a910c55b4f928acd0a5e4b346673 P 68b666b388e54d3a93ae86c6098c7b44: Bootstrap starting.
I20260812 06:37:40.299199 17076 tablet_bootstrap.cc:654] T c0b0a910c55b4f928acd0a5e4b346673 P d41d668ab84c421381071558fe019747: Neither blocks nor log segments found. Creating new log.
I20260812 06:37:40.299952 17077 tablet_bootstrap.cc:654] T c0b0a910c55b4f928acd0a5e4b346673 P d42658cb732d4733b992c5e7a7540576: Neither blocks nor log segments found. Creating new log.
I20260812 06:37:40.300053 17076 log.cc:826] T c0b0a910c55b4f928acd0a5e4b346673 P d41d668ab84c421381071558fe019747: Log is configured to *not* fsync() on all Append() calls
I20260812 06:37:40.300459 17079 tablet_bootstrap.cc:654] T c0b0a910c55b4f928acd0a5e4b346673 P 68b666b388e54d3a93ae86c6098c7b44: Neither blocks nor log segments found. Creating new log.
I20260812 06:37:40.300738 17077 log.cc:826] T c0b0a910c55b4f928acd0a5e4b346673 P d42658cb732d4733b992c5e7a7540576: Log is configured to *not* fsync() on all Append() calls
I20260812 06:37:40.301203 17079 log.cc:826] T c0b0a910c55b4f928acd0a5e4b346673 P 68b666b388e54d3a93ae86c6098c7b44: Log is configured to *not* fsync() on all Append() calls
I20260812 06:37:40.301936 17076 tablet_bootstrap.cc:492] T c0b0a910c55b4f928acd0a5e4b346673 P d41d668ab84c421381071558fe019747: No bootstrap required, opened a new log
I20260812 06:37:40.302156 17076 ts_tablet_manager.cc:1403] T c0b0a910c55b4f928acd0a5e4b346673 P d41d668ab84c421381071558fe019747: Time spent bootstrapping tablet: real 0.006s	user 0.004s	sys 0.000s
I20260812 06:37:40.302374 17077 tablet_bootstrap.cc:492] T c0b0a910c55b4f928acd0a5e4b346673 P d42658cb732d4733b992c5e7a7540576: No bootstrap required, opened a new log
I20260812 06:37:40.302551 17077 ts_tablet_manager.cc:1403] T c0b0a910c55b4f928acd0a5e4b346673 P d42658cb732d4733b992c5e7a7540576: Time spent bootstrapping tablet: real 0.005s	user 0.004s	sys 0.000s
I20260812 06:37:40.302724 17079 tablet_bootstrap.cc:492] T c0b0a910c55b4f928acd0a5e4b346673 P 68b666b388e54d3a93ae86c6098c7b44: No bootstrap required, opened a new log
I20260812 06:37:40.302904 17079 ts_tablet_manager.cc:1403] T c0b0a910c55b4f928acd0a5e4b346673 P 68b666b388e54d3a93ae86c6098c7b44: Time spent bootstrapping tablet: real 0.004s	user 0.004s	sys 0.000s
I20260812 06:37:40.305872 17076 raft_consensus.cc:359] T c0b0a910c55b4f928acd0a5e4b346673 P d41d668ab84c421381071558fe019747 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68b666b388e54d3a93ae86c6098c7b44" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 40995 } } peers { permanent_uuid: "d42658cb732d4733b992c5e7a7540576" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 38765 } } peers { permanent_uuid: "d41d668ab84c421381071558fe019747" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 36879 } }
I20260812 06:37:40.306105 17076 raft_consensus.cc:385] T c0b0a910c55b4f928acd0a5e4b346673 P d41d668ab84c421381071558fe019747 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:37:40.305964 17079 raft_consensus.cc:359] T c0b0a910c55b4f928acd0a5e4b346673 P 68b666b388e54d3a93ae86c6098c7b44 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68b666b388e54d3a93ae86c6098c7b44" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 40995 } } peers { permanent_uuid: "d42658cb732d4733b992c5e7a7540576" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 38765 } } peers { permanent_uuid: "d41d668ab84c421381071558fe019747" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 36879 } }
I20260812 06:37:40.306169 17076 raft_consensus.cc:740] T c0b0a910c55b4f928acd0a5e4b346673 P d41d668ab84c421381071558fe019747 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d41d668ab84c421381071558fe019747, State: Initialized, Role: FOLLOWER
I20260812 06:37:40.306185 17079 raft_consensus.cc:385] T c0b0a910c55b4f928acd0a5e4b346673 P 68b666b388e54d3a93ae86c6098c7b44 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:37:40.306275 17079 raft_consensus.cc:740] T c0b0a910c55b4f928acd0a5e4b346673 P 68b666b388e54d3a93ae86c6098c7b44 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 68b666b388e54d3a93ae86c6098c7b44, State: Initialized, Role: FOLLOWER
I20260812 06:37:40.306684 17076 consensus_queue.cc:260] T c0b0a910c55b4f928acd0a5e4b346673 P d41d668ab84c421381071558fe019747 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68b666b388e54d3a93ae86c6098c7b44" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 40995 } } peers { permanent_uuid: "d42658cb732d4733b992c5e7a7540576" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 38765 } } peers { permanent_uuid: "d41d668ab84c421381071558fe019747" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 36879 } }
I20260812 06:37:40.306711 17079 consensus_queue.cc:260] T c0b0a910c55b4f928acd0a5e4b346673 P 68b666b388e54d3a93ae86c6098c7b44 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68b666b388e54d3a93ae86c6098c7b44" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 40995 } } peers { permanent_uuid: "d42658cb732d4733b992c5e7a7540576" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 38765 } } peers { permanent_uuid: "d41d668ab84c421381071558fe019747" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 36879 } }
I20260812 06:37:40.307101 17077 raft_consensus.cc:359] T c0b0a910c55b4f928acd0a5e4b346673 P d42658cb732d4733b992c5e7a7540576 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68b666b388e54d3a93ae86c6098c7b44" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 40995 } } peers { permanent_uuid: "d42658cb732d4733b992c5e7a7540576" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 38765 } } peers { permanent_uuid: "d41d668ab84c421381071558fe019747" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 36879 } }
I20260812 06:37:40.307281 17077 raft_consensus.cc:385] T c0b0a910c55b4f928acd0a5e4b346673 P d42658cb732d4733b992c5e7a7540576 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 06:37:40.307334 17077 raft_consensus.cc:740] T c0b0a910c55b4f928acd0a5e4b346673 P d42658cb732d4733b992c5e7a7540576 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d42658cb732d4733b992c5e7a7540576, State: Initialized, Role: FOLLOWER
I20260812 06:37:40.307586 17076 ts_tablet_manager.cc:1434] T c0b0a910c55b4f928acd0a5e4b346673 P d41d668ab84c421381071558fe019747: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260812 06:37:40.307595 17079 ts_tablet_manager.cc:1434] T c0b0a910c55b4f928acd0a5e4b346673 P 68b666b388e54d3a93ae86c6098c7b44: Time spent starting tablet: real 0.005s	user 0.000s	sys 0.004s
I20260812 06:37:40.307763 17077 consensus_queue.cc:260] T c0b0a910c55b4f928acd0a5e4b346673 P d42658cb732d4733b992c5e7a7540576 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68b666b388e54d3a93ae86c6098c7b44" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 40995 } } peers { permanent_uuid: "d42658cb732d4733b992c5e7a7540576" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 38765 } } peers { permanent_uuid: "d41d668ab84c421381071558fe019747" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 36879 } }
I20260812 06:37:40.307956 16584 heartbeater.cc:499] Master 127.4.177.62:41039 was elected leader, sending a full tablet report...
I20260812 06:37:40.308383 17051 heartbeater.cc:499] Master 127.4.177.62:41039 was elected leader, sending a full tablet report...
I20260812 06:37:40.308383 16827 heartbeater.cc:499] Master 127.4.177.62:41039 was elected leader, sending a full tablet report...
I20260812 06:37:40.309556 17077 ts_tablet_manager.cc:1434] T c0b0a910c55b4f928acd0a5e4b346673 P d42658cb732d4733b992c5e7a7540576: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.000s
W20260812 06:37:40.340562 17053 tablet.cc:2406] T c0b0a910c55b4f928acd0a5e4b346673 P d41d668ab84c421381071558fe019747: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 06:37:40.366294 16828 tablet.cc:2406] T c0b0a910c55b4f928acd0a5e4b346673 P d42658cb732d4733b992c5e7a7540576: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260812 06:37:40.375489 16585 tablet.cc:2406] T c0b0a910c55b4f928acd0a5e4b346673 P 68b666b388e54d3a93ae86c6098c7b44: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 06:37:40.510651 17087 raft_consensus.cc:493] T c0b0a910c55b4f928acd0a5e4b346673 P d42658cb732d4733b992c5e7a7540576 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 06:37:40.510864 17087 raft_consensus.cc:515] T c0b0a910c55b4f928acd0a5e4b346673 P d42658cb732d4733b992c5e7a7540576 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68b666b388e54d3a93ae86c6098c7b44" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 40995 } } peers { permanent_uuid: "d42658cb732d4733b992c5e7a7540576" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 38765 } } peers { permanent_uuid: "d41d668ab84c421381071558fe019747" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 36879 } }
I20260812 06:37:40.511986 17087 leader_election.cc:290] T c0b0a910c55b4f928acd0a5e4b346673 P d42658cb732d4733b992c5e7a7540576 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 68b666b388e54d3a93ae86c6098c7b44 (127.4.177.1:40995), d41d668ab84c421381071558fe019747 (127.4.177.3:36879)
I20260812 06:37:40.516729 16502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b0a910c55b4f928acd0a5e4b346673" candidate_uuid: "d42658cb732d4733b992c5e7a7540576" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68b666b388e54d3a93ae86c6098c7b44" is_pre_election: true
I20260812 06:37:40.517050 16502 raft_consensus.cc:2468] T c0b0a910c55b4f928acd0a5e4b346673 P 68b666b388e54d3a93ae86c6098c7b44 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d42658cb732d4733b992c5e7a7540576 in term 0.
I20260812 06:37:40.517138 16963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b0a910c55b4f928acd0a5e4b346673" candidate_uuid: "d42658cb732d4733b992c5e7a7540576" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d41d668ab84c421381071558fe019747" is_pre_election: true
I20260812 06:37:40.517457 16963 raft_consensus.cc:2468] T c0b0a910c55b4f928acd0a5e4b346673 P d41d668ab84c421381071558fe019747 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d42658cb732d4733b992c5e7a7540576 in term 0.
I20260812 06:37:40.517612 16614 leader_election.cc:304] T c0b0a910c55b4f928acd0a5e4b346673 P d42658cb732d4733b992c5e7a7540576 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 68b666b388e54d3a93ae86c6098c7b44, d42658cb732d4733b992c5e7a7540576; no voters: 
I20260812 06:37:40.517896 17087 raft_consensus.cc:2804] T c0b0a910c55b4f928acd0a5e4b346673 P d42658cb732d4733b992c5e7a7540576 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 06:37:40.517982 17087 raft_consensus.cc:493] T c0b0a910c55b4f928acd0a5e4b346673 P d42658cb732d4733b992c5e7a7540576 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 06:37:40.518028 17087 raft_consensus.cc:3060] T c0b0a910c55b4f928acd0a5e4b346673 P d42658cb732d4733b992c5e7a7540576 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:37:40.519111 17087 raft_consensus.cc:515] T c0b0a910c55b4f928acd0a5e4b346673 P d42658cb732d4733b992c5e7a7540576 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68b666b388e54d3a93ae86c6098c7b44" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 40995 } } peers { permanent_uuid: "d42658cb732d4733b992c5e7a7540576" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 38765 } } peers { permanent_uuid: "d41d668ab84c421381071558fe019747" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 36879 } }
I20260812 06:37:40.519480 17087 leader_election.cc:290] T c0b0a910c55b4f928acd0a5e4b346673 P d42658cb732d4733b992c5e7a7540576 [CANDIDATE]: Term 1 election: Requested vote from peers 68b666b388e54d3a93ae86c6098c7b44 (127.4.177.1:40995), d41d668ab84c421381071558fe019747 (127.4.177.3:36879)
I20260812 06:37:40.519841 16963 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b0a910c55b4f928acd0a5e4b346673" candidate_uuid: "d42658cb732d4733b992c5e7a7540576" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d41d668ab84c421381071558fe019747"
I20260812 06:37:40.519824 16502 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b0a910c55b4f928acd0a5e4b346673" candidate_uuid: "d42658cb732d4733b992c5e7a7540576" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "68b666b388e54d3a93ae86c6098c7b44"
I20260812 06:37:40.519956 16963 raft_consensus.cc:3060] T c0b0a910c55b4f928acd0a5e4b346673 P d41d668ab84c421381071558fe019747 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:37:40.519958 16502 raft_consensus.cc:3060] T c0b0a910c55b4f928acd0a5e4b346673 P 68b666b388e54d3a93ae86c6098c7b44 [term 0 FOLLOWER]: Advancing to term 1
I20260812 06:37:40.521180 16963 raft_consensus.cc:2468] T c0b0a910c55b4f928acd0a5e4b346673 P d41d668ab84c421381071558fe019747 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d42658cb732d4733b992c5e7a7540576 in term 1.
I20260812 06:37:40.521550 16616 leader_election.cc:304] T c0b0a910c55b4f928acd0a5e4b346673 P d42658cb732d4733b992c5e7a7540576 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d41d668ab84c421381071558fe019747, d42658cb732d4733b992c5e7a7540576; no voters: 
I20260812 06:37:40.521626 16502 raft_consensus.cc:2468] T c0b0a910c55b4f928acd0a5e4b346673 P 68b666b388e54d3a93ae86c6098c7b44 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d42658cb732d4733b992c5e7a7540576 in term 1.
I20260812 06:37:40.521785 17087 raft_consensus.cc:2804] T c0b0a910c55b4f928acd0a5e4b346673 P d42658cb732d4733b992c5e7a7540576 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 06:37:40.522001 17087 raft_consensus.cc:697] T c0b0a910c55b4f928acd0a5e4b346673 P d42658cb732d4733b992c5e7a7540576 [term 1 LEADER]: Becoming Leader. State: Replica: d42658cb732d4733b992c5e7a7540576, State: Running, Role: LEADER
I20260812 06:37:40.522395 17087 consensus_queue.cc:237] T c0b0a910c55b4f928acd0a5e4b346673 P d42658cb732d4733b992c5e7a7540576 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68b666b388e54d3a93ae86c6098c7b44" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 40995 } } peers { permanent_uuid: "d42658cb732d4733b992c5e7a7540576" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 38765 } } peers { permanent_uuid: "d41d668ab84c421381071558fe019747" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 36879 } }
I20260812 06:37:40.525424 16268 catalog_manager.cc:5719] T c0b0a910c55b4f928acd0a5e4b346673 P d42658cb732d4733b992c5e7a7540576 reported cstate change: term changed from 0 to 1, leader changed from <none> to d42658cb732d4733b992c5e7a7540576 (127.4.177.2). New cstate: current_term: 1 leader_uuid: "d42658cb732d4733b992c5e7a7540576" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68b666b388e54d3a93ae86c6098c7b44" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 40995 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d42658cb732d4733b992c5e7a7540576" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 38765 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d41d668ab84c421381071558fe019747" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 36879 } health_report { overall_health: UNKNOWN } } }
I20260812 06:37:40.575984  4804 external_mini_cluster.cc:955] 3 TS(s) registered with all masters
I20260812 06:37:40.577600  4804 ts_itest-base.cc:246] Waiting for 1 tablets on tserver 68b666b388e54d3a93ae86c6098c7b44 to finish bootstrapping
I20260812 06:37:40.583045  4804 ts_itest-base.cc:246] Waiting for 1 tablets on tserver d42658cb732d4733b992c5e7a7540576 to finish bootstrapping
I20260812 06:37:40.588824  4804 ts_itest-base.cc:246] Waiting for 1 tablets on tserver d41d668ab84c421381071558fe019747 to finish bootstrapping
I20260812 06:37:40.605055  4804 cluster_itest_util.cc:258] Not converged past 1 yet: 1.1 0.0 0.0
I20260812 06:37:40.708166  4804 cluster_itest_util.cc:258] Not converged past 1 yet: 1.1 0.0 0.0
I20260812 06:37:40.911374  4804 cluster_itest_util.cc:258] Not converged past 1 yet: 1.1 0.0 0.0
I20260812 06:37:40.979192 17087 consensus_queue.cc:1048] T c0b0a910c55b4f928acd0a5e4b346673 P d42658cb732d4733b992c5e7a7540576 [LEADER]: Connected to new peer: Peer: permanent_uuid: "68b666b388e54d3a93ae86c6098c7b44" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 40995 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 06:37:40.984752 17090 consensus_queue.cc:1048] T c0b0a910c55b4f928acd0a5e4b346673 P d42658cb732d4733b992c5e7a7540576 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d41d668ab84c421381071558fe019747" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 36879 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 06:37:41.237277  4804 test_workload.cc:405] TestWorkload: Skipping table creation because table TestTable already exists
I20260812 06:37:41.237676  4804 raft_consensus-itest-base.cc:268] Waiting until we've written at least 4MB...
I20260812 06:37:41.336043  4804 raft_consensus-itest-base.cc:274] Waiting for log GC on d42658cb732d4733b992c5e7a7540576
I20260812 06:37:41.370973 16828 maintenance_manager.cc:419] P d42658cb732d4733b992c5e7a7540576: Scheduling FlushMRSOp(c0b0a910c55b4f928acd0a5e4b346673): 4602240 bytes log retention, and flush 7160073 bytes memory
I20260812 06:37:41.379936 16585 maintenance_manager.cc:419] P 68b666b388e54d3a93ae86c6098c7b44: Scheduling FlushMRSOp(c0b0a910c55b4f928acd0a5e4b346673): 4602032 bytes log retention, and flush 7160073 bytes memory
I20260812 06:37:41.437095 16619 maintenance_manager.cc:643] P d42658cb732d4733b992c5e7a7540576: FlushMRSOp(c0b0a910c55b4f928acd0a5e4b346673) complete. Timing: real 0.066s	user 0.059s	sys 0.004s Metrics: {"bytes_written":136065,"cfile_init":1,"compiler_manager_pool.queue_time_us":292,"drs_written":1,"lbm_read_time_us":71,"lbm_reads_lt_1ms":4,"lbm_write_time_us":690,"lbm_writes_lt_1ms":23,"peak_mem_usage":0,"rows_written":41,"thread_start_us":302,"threads_started":2}
I20260812 06:37:41.438143 16828 maintenance_manager.cc:419] P d42658cb732d4733b992c5e7a7540576: Scheduling LogGCOp(c0b0a910c55b4f928acd0a5e4b346673): free 2761328 bytes of WAL
I20260812 06:37:41.438431 16619 log_reader.cc:385] T c0b0a910c55b4f928acd0a5e4b346673: removed 3 log segments from log reader
I20260812 06:37:41.438503 16619 log.cc:1079] T c0b0a910c55b4f928acd0a5e4b346673 P d42658cb732d4733b992c5e7a7540576: Deleting log segment in path: /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.EncryptionEnabled_RaftConsensusParamEncryptionITest.TestFollowerFallsBehindLeaderGC_0.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-1/wal/wals/c0b0a910c55b4f928acd0a5e4b346673/wal-000000001 (ops 1-8)
I20260812 06:37:41.438581 16619 log.cc:1079] T c0b0a910c55b4f928acd0a5e4b346673 P d42658cb732d4733b992c5e7a7540576: Deleting log segment in path: /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.EncryptionEnabled_RaftConsensusParamEncryptionITest.TestFollowerFallsBehindLeaderGC_0.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-1/wal/wals/c0b0a910c55b4f928acd0a5e4b346673/wal-000000002 (ops 9-15)
I20260812 06:37:41.438627 16619 log.cc:1079] T c0b0a910c55b4f928acd0a5e4b346673 P d42658cb732d4733b992c5e7a7540576: Deleting log segment in path: /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.EncryptionEnabled_RaftConsensusParamEncryptionITest.TestFollowerFallsBehindLeaderGC_0.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-1/wal/wals/c0b0a910c55b4f928acd0a5e4b346673/wal-000000003 (ops 16-22)
I20260812 06:37:41.439711 16619 maintenance_manager.cc:643] P d42658cb732d4733b992c5e7a7540576: LogGCOp(c0b0a910c55b4f928acd0a5e4b346673) complete. Timing: real 0.001s	user 0.000s	sys 0.000s Metrics: {}
I20260812 06:37:41.440016 16828 maintenance_manager.cc:419] P d42658cb732d4733b992c5e7a7540576: Scheduling UndoDeltaBlockGCOp(c0b0a910c55b4f928acd0a5e4b346673): 595 bytes on disk
I20260812 06:37:41.440836 16619 maintenance_manager.cc:643] P d42658cb732d4733b992c5e7a7540576: UndoDeltaBlockGCOp(c0b0a910c55b4f928acd0a5e4b346673) complete. Timing: real 0.000s	user 0.000s	sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":63,"lbm_reads_lt_1ms":4}
I20260812 06:37:41.448808  4804 raft_consensus-itest-base.cc:279] Log GC complete on d42658cb732d4733b992c5e7a7540576
I20260812 06:37:41.449314 16396 maintenance_manager.cc:643] P 68b666b388e54d3a93ae86c6098c7b44: FlushMRSOp(c0b0a910c55b4f928acd0a5e4b346673) complete. Timing: real 0.069s	user 0.052s	sys 0.012s Metrics: {"bytes_written":136065,"cfile_init":1,"compiler_manager_pool.queue_time_us":386,"dirs.queue_time_us":265,"dirs.run_cpu_time_us":180,"dirs.run_wall_time_us":651,"drs_written":1,"lbm_read_time_us":70,"lbm_reads_lt_1ms":4,"lbm_write_time_us":730,"lbm_writes_lt_1ms":23,"peak_mem_usage":0,"rows_written":41,"thread_start_us":312,"threads_started":2}
I20260812 06:37:41.450345 16585 maintenance_manager.cc:419] P 68b666b388e54d3a93ae86c6098c7b44: Scheduling LogGCOp(c0b0a910c55b4f928acd0a5e4b346673): free 2761292 bytes of WAL
I20260812 06:37:41.450634 16396 log_reader.cc:385] T c0b0a910c55b4f928acd0a5e4b346673: removed 3 log segments from log reader
I20260812 06:37:41.450698 16396 log.cc:1079] T c0b0a910c55b4f928acd0a5e4b346673 P 68b666b388e54d3a93ae86c6098c7b44: Deleting log segment in path: /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.EncryptionEnabled_RaftConsensusParamEncryptionITest.TestFollowerFallsBehindLeaderGC_0.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-0/wal/wals/c0b0a910c55b4f928acd0a5e4b346673/wal-000000001 (ops 1-8)
I20260812 06:37:41.450760 16396 log.cc:1079] T c0b0a910c55b4f928acd0a5e4b346673 P 68b666b388e54d3a93ae86c6098c7b44: Deleting log segment in path: /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.EncryptionEnabled_RaftConsensusParamEncryptionITest.TestFollowerFallsBehindLeaderGC_0.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-0/wal/wals/c0b0a910c55b4f928acd0a5e4b346673/wal-000000002 (ops 9-15)
I20260812 06:37:41.450798 16396 log.cc:1079] T c0b0a910c55b4f928acd0a5e4b346673 P 68b666b388e54d3a93ae86c6098c7b44: Deleting log segment in path: /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.EncryptionEnabled_RaftConsensusParamEncryptionITest.TestFollowerFallsBehindLeaderGC_0.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-0/wal/wals/c0b0a910c55b4f928acd0a5e4b346673/wal-000000003 (ops 16-22)
I20260812 06:37:41.451814 16396 maintenance_manager.cc:643] P 68b666b388e54d3a93ae86c6098c7b44: LogGCOp(c0b0a910c55b4f928acd0a5e4b346673) complete. Timing: real 0.001s	user 0.000s	sys 0.000s Metrics: {}
I20260812 06:37:41.452109 16585 maintenance_manager.cc:419] P 68b666b388e54d3a93ae86c6098c7b44: Scheduling UndoDeltaBlockGCOp(c0b0a910c55b4f928acd0a5e4b346673): 595 bytes on disk
I20260812 06:37:41.452623 16396 maintenance_manager.cc:643] P 68b666b388e54d3a93ae86c6098c7b44: UndoDeltaBlockGCOp(c0b0a910c55b4f928acd0a5e4b346673) complete. Timing: real 0.000s	user 0.000s	sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":66,"lbm_reads_lt_1ms":4}
I20260812 06:37:43.450196  4804 raft_consensus-itest-base.cc:295] Servers converged with original term 1
I20260812 06:37:43.450294  4804 raft_consensus-itest-base.cc:300] Resuming d41d668ab84c421381071558fe019747
I20260812 06:37:43.451270 17106 raft_consensus.cc:493] T c0b0a910c55b4f928acd0a5e4b346673 P d41d668ab84c421381071558fe019747 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d42658cb732d4733b992c5e7a7540576)
I20260812 06:37:43.451417 17106 raft_consensus.cc:515] T c0b0a910c55b4f928acd0a5e4b346673 P d41d668ab84c421381071558fe019747 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "68b666b388e54d3a93ae86c6098c7b44" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 40995 } } peers { permanent_uuid: "d42658cb732d4733b992c5e7a7540576" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 38765 } } peers { permanent_uuid: "d41d668ab84c421381071558fe019747" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 36879 } }
I20260812 06:37:43.452656 17106 leader_election.cc:290] T c0b0a910c55b4f928acd0a5e4b346673 P d41d668ab84c421381071558fe019747 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 68b666b388e54d3a93ae86c6098c7b44 (127.4.177.1:40995), d42658cb732d4733b992c5e7a7540576 (127.4.177.2:38765)
I20260812 06:37:43.456269 17158 consensus_queue.cc:753] T c0b0a910c55b4f928acd0a5e4b346673 P d42658cb732d4733b992c5e7a7540576 [LEADER]: The logs necessary to catch up peer d41d668ab84c421381071558fe019747 have been garbage collected. The follower will never be able to catch up (Not found: failed to read ops 3..36: Segment 1 which contained index 3 has been GCed)
I20260812 06:37:43.456348 17158 consensus_queue.cc:579] T c0b0a910c55b4f928acd0a5e4b346673 P d42658cb732d4733b992c5e7a7540576 [LEADER]: The logs necessary to catch up peer d41d668ab84c421381071558fe019747 have been garbage collected. The replica will never be able to catch up
I20260812 06:37:43.459574 16501 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b0a910c55b4f928acd0a5e4b346673" candidate_uuid: "d41d668ab84c421381071558fe019747" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "68b666b388e54d3a93ae86c6098c7b44" is_pre_election: true
I20260812 06:37:43.459621 16741 consensus_queue.cc:237] T c0b0a910c55b4f928acd0a5e4b346673 P d42658cb732d4733b992c5e7a7540576 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 42, Committed index: 42, Last appended: 1.42, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 43 OBSOLETE_local: false peers { permanent_uuid: "68b666b388e54d3a93ae86c6098c7b44" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 40995 } } peers { permanent_uuid: "d42658cb732d4733b992c5e7a7540576" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 38765 } }
I20260812 06:37:43.460744 16501 raft_consensus.cc:1275] T c0b0a910c55b4f928acd0a5e4b346673 P 68b666b388e54d3a93ae86c6098c7b44 [term 1 FOLLOWER]: Refusing update from remote peer d42658cb732d4733b992c5e7a7540576: Log matching property violated. Preceding OpId in replica: term: 1 index: 42. Preceding OpId from leader: term: 1 index: 43. (index mismatch)
I20260812 06:37:43.461205 17176 consensus_queue.cc:1048] T c0b0a910c55b4f928acd0a5e4b346673 P d42658cb732d4733b992c5e7a7540576 [LEADER]: Connected to new peer: Peer: permanent_uuid: "68b666b388e54d3a93ae86c6098c7b44" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 40995 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 43, Last known committed idx: 42, Time since last communication: 0.000s
I20260812 06:37:43.461416 16741 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0b0a910c55b4f928acd0a5e4b346673" candidate_uuid: "d41d668ab84c421381071558fe019747" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d42658cb732d4733b992c5e7a7540576" is_pre_election: true
I20260812 06:37:43.461928 16856 leader_election.cc:304] T c0b0a910c55b4f928acd0a5e4b346673 P d41d668ab84c421381071558fe019747 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d41d668ab84c421381071558fe019747; no voters: 68b666b388e54d3a93ae86c6098c7b44, d42658cb732d4733b992c5e7a7540576
I20260812 06:37:43.462296 17106 raft_consensus.cc:2749] T c0b0a910c55b4f928acd0a5e4b346673 P d41d668ab84c421381071558fe019747 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260812 06:37:43.465617 17158 raft_consensus.cc:2955] T c0b0a910c55b4f928acd0a5e4b346673 P d42658cb732d4733b992c5e7a7540576 [term 1 LEADER]: Committing config change with OpId 1.43: config changed from index -1 to 43, VOTER d41d668ab84c421381071558fe019747 (127.4.177.3) evicted. New config: { opid_index: 43 OBSOLETE_local: false peers { permanent_uuid: "68b666b388e54d3a93ae86c6098c7b44" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 40995 } } peers { permanent_uuid: "d42658cb732d4733b992c5e7a7540576" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 38765 } } }
I20260812 06:37:43.466244 16501 raft_consensus.cc:2955] T c0b0a910c55b4f928acd0a5e4b346673 P 68b666b388e54d3a93ae86c6098c7b44 [term 1 FOLLOWER]: Committing config change with OpId 1.43: config changed from index -1 to 43, VOTER d41d668ab84c421381071558fe019747 (127.4.177.3) evicted. New config: { opid_index: 43 OBSOLETE_local: false peers { permanent_uuid: "68b666b388e54d3a93ae86c6098c7b44" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 40995 } } peers { permanent_uuid: "d42658cb732d4733b992c5e7a7540576" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 38765 } } }
I20260812 06:37:43.467618 16243 catalog_manager.cc:5232] ChangeConfig:REMOVE_PEER RPC for tablet c0b0a910c55b4f928acd0a5e4b346673 with cas_config_opid_index -1: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260812 06:37:43.468503 16265 catalog_manager.cc:5719] T c0b0a910c55b4f928acd0a5e4b346673 P d42658cb732d4733b992c5e7a7540576 reported cstate change: config changed from index -1 to 43, VOTER d41d668ab84c421381071558fe019747 (127.4.177.3) evicted. New cstate: current_term: 1 leader_uuid: "d42658cb732d4733b992c5e7a7540576" committed_config { opid_index: 43 OBSOLETE_local: false peers { permanent_uuid: "68b666b388e54d3a93ae86c6098c7b44" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 40995 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d42658cb732d4733b992c5e7a7540576" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 38765 } health_report { overall_health: HEALTHY } } }
I20260812 06:37:43.471571 16931 tablet_service.cc:1558] Processing DeleteTablet for tablet c0b0a910c55b4f928acd0a5e4b346673 with delete_type TABLET_DATA_TOMBSTONED (TS d41d668ab84c421381071558fe019747 not found in new config with opid_index 43) from {username='slave'} at 127.0.0.1:50542
I20260812 06:37:43.472116 17188 tablet_replica.cc:333] T c0b0a910c55b4f928acd0a5e4b346673 P d41d668ab84c421381071558fe019747: stopping tablet replica
I20260812 06:37:43.472144 16741 consensus_queue.cc:237] T c0b0a910c55b4f928acd0a5e4b346673 P d42658cb732d4733b992c5e7a7540576 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 43, Committed index: 43, Last appended: 1.43, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 44 OBSOLETE_local: false peers { permanent_uuid: "68b666b388e54d3a93ae86c6098c7b44" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 40995 } } peers { permanent_uuid: "d42658cb732d4733b992c5e7a7540576" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 38765 } } peers { permanent_uuid: "d41d668ab84c421381071558fe019747" member_type: NON_VOTER last_known_addr { host: "127.4.177.3" port: 36879 } attrs { promote: true } }
I20260812 06:37:43.472446 17188 raft_consensus.cc:2243] T c0b0a910c55b4f928acd0a5e4b346673 P d41d668ab84c421381071558fe019747 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 06:37:43.472644 17188 pending_rounds.cc:70] T c0b0a910c55b4f928acd0a5e4b346673 P d41d668ab84c421381071558fe019747: Trying to abort 1 pending ops.
I20260812 06:37:43.473685 16501 raft_consensus.cc:1275] T c0b0a910c55b4f928acd0a5e4b346673 P 68b666b388e54d3a93ae86c6098c7b44 [term 1 FOLLOWER]: Refusing update from remote peer d42658cb732d4733b992c5e7a7540576: Log matching property violated. Preceding OpId in replica: term: 1 index: 43. Preceding OpId from leader: term: 1 index: 44. (index mismatch)
W20260812 06:37:43.473948 16616 consensus_peers.cc:597] T c0b0a910c55b4f928acd0a5e4b346673 P d42658cb732d4733b992c5e7a7540576 -> Peer d41d668ab84c421381071558fe019747 (127.4.177.3:36879): Couldn't send request to peer d41d668ab84c421381071558fe019747. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: STOPPING. This is attempt 1: this message will repeat every 5th retry.
I20260812 06:37:43.474155 17158 consensus_queue.cc:1048] T c0b0a910c55b4f928acd0a5e4b346673 P d42658cb732d4733b992c5e7a7540576 [LEADER]: Connected to new peer: Peer: permanent_uuid: "68b666b388e54d3a93ae86c6098c7b44" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 40995 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 44, Last known committed idx: 43, Time since last communication: 0.000s
I20260812 06:37:43.472740 17188 pending_rounds.cc:77] T c0b0a910c55b4f928acd0a5e4b346673 P d41d668ab84c421381071558fe019747: Aborting op as it isn't in flight: id { term: 1 index: 2 } timestamp: 7317572244479225856 op_type: WRITE_OP write_request { tablet_id: "c0b0a910c55b4f928acd0a5e4b346673" schema { columns { name: "key" type: INT32 is_key: true is_nullable: false immutable: false } columns { name: "int_val" type: INT32 is_key: false is_nullable: false immutable: false } columns { name: "string_val" type: STRING is_key: false is_nullable: true immutable: false } } row_operations { rows: "<redacted>""\001\007\000\022z\354\004o\312\346\315\000\000\000\000\000\000\000\000\000\000\002\000\000\000\000\000" indirect_data: "<redacted>""000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
I20260812 06:37:43.475613 17188 raft_consensus.cc:2272] T c0b0a910c55b4f928acd0a5e4b346673 P d41d668ab84c421381071558fe019747 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 06:37:43.475771 17176 raft_consensus.cc:2955] T c0b0a910c55b4f928acd0a5e4b346673 P d42658cb732d4733b992c5e7a7540576 [term 1 LEADER]: Committing config change with OpId 1.44: config changed from index 43 to 44, NON_VOTER d41d668ab84c421381071558fe019747 (127.4.177.3) added. New config: { opid_index: 44 OBSOLETE_local: false peers { permanent_uuid: "68b666b388e54d3a93ae86c6098c7b44" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 40995 } } peers { permanent_uuid: "d42658cb732d4733b992c5e7a7540576" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 38765 } } peers { permanent_uuid: "d41d668ab84c421381071558fe019747" member_type: NON_VOTER last_known_addr { host: "127.4.177.3" port: 36879 } attrs { promote: true } } }
I20260812 06:37:43.476240 16501 raft_consensus.cc:2955] T c0b0a910c55b4f928acd0a5e4b346673 P 68b666b388e54d3a93ae86c6098c7b44 [term 1 FOLLOWER]: Committing config change with OpId 1.44: config changed from index 43 to 44, NON_VOTER d41d668ab84c421381071558fe019747 (127.4.177.3) added. New config: { opid_index: 44 OBSOLETE_local: false peers { permanent_uuid: "68b666b388e54d3a93ae86c6098c7b44" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 40995 } } peers { permanent_uuid: "d42658cb732d4733b992c5e7a7540576" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 38765 } } peers { permanent_uuid: "d41d668ab84c421381071558fe019747" member_type: NON_VOTER last_known_addr { host: "127.4.177.3" port: 36879 } attrs { promote: true } } }
I20260812 06:37:43.476897 17188 ts_tablet_manager.cc:1916] T c0b0a910c55b4f928acd0a5e4b346673 P d41d668ab84c421381071558fe019747: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260812 06:37:43.478118 16265 catalog_manager.cc:5719] T c0b0a910c55b4f928acd0a5e4b346673 P d42658cb732d4733b992c5e7a7540576 reported cstate change: config changed from index 43 to 44, NON_VOTER d41d668ab84c421381071558fe019747 (127.4.177.3) added. New cstate: current_term: 1 leader_uuid: "d42658cb732d4733b992c5e7a7540576" committed_config { opid_index: 44 OBSOLETE_local: false peers { permanent_uuid: "68b666b388e54d3a93ae86c6098c7b44" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 40995 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d42658cb732d4733b992c5e7a7540576" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 38765 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d41d668ab84c421381071558fe019747" member_type: NON_VOTER last_known_addr { host: "127.4.177.3" port: 36879 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260812 06:37:43.478535 16243 catalog_manager.cc:5232] ChangeConfig:ADD_PEER:NON_VOTER RPC for tablet c0b0a910c55b4f928acd0a5e4b346673 with cas_config_opid_index 43: ChangeConfig:ADD_PEER:NON_VOTER succeeded (attempt 1)
I20260812 06:37:43.479598 17188 ts_tablet_manager.cc:1929] T c0b0a910c55b4f928acd0a5e4b346673 P d41d668ab84c421381071558fe019747: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.2
I20260812 06:37:43.479717 17188 log.cc:1199] T c0b0a910c55b4f928acd0a5e4b346673 P d41d668ab84c421381071558fe019747: Deleting WAL directory at /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.EncryptionEnabled_RaftConsensusParamEncryptionITest.TestFollowerFallsBehindLeaderGC_0.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-2/wal/wals/c0b0a910c55b4f928acd0a5e4b346673
I20260812 06:37:43.480368 16244 catalog_manager.cc:5050] TS d41d668ab84c421381071558fe019747 (127.4.177.3:36879): tablet c0b0a910c55b4f928acd0a5e4b346673 (table TestTable [id=1d35b3f1c7c34c218d722c94878d0562]) successfully deleted
I20260812 06:37:44.619153 17191 ts_tablet_manager.cc:933] T c0b0a910c55b4f928acd0a5e4b346673 P d41d668ab84c421381071558fe019747: Initiating tablet copy from peer d42658cb732d4733b992c5e7a7540576 (127.4.177.2:38765)
I20260812 06:37:44.619823 17191 tablet_copy_client.cc:323] T c0b0a910c55b4f928acd0a5e4b346673 P d41d668ab84c421381071558fe019747: tablet copy: Beginning tablet copy session from remote peer at address 127.4.177.2:38765
I20260812 06:37:44.620779 16774 tablet_copy_service.cc:140] P d42658cb732d4733b992c5e7a7540576: Received BeginTabletCopySession request for tablet c0b0a910c55b4f928acd0a5e4b346673 from peer d41d668ab84c421381071558fe019747 ({username='slave'} at 127.4.177.3:53301)
I20260812 06:37:44.620976 16774 tablet_copy_service.cc:161] P d42658cb732d4733b992c5e7a7540576: Beginning new tablet copy session on tablet c0b0a910c55b4f928acd0a5e4b346673 from peer d41d668ab84c421381071558fe019747 at {username='slave'} at 127.4.177.3:53301: session id = d41d668ab84c421381071558fe019747-c0b0a910c55b4f928acd0a5e4b346673
I20260812 06:37:44.622570 16774 tablet_copy_source_session.cc:215] T c0b0a910c55b4f928acd0a5e4b346673 P d42658cb732d4733b992c5e7a7540576: Tablet Copy: opened 5 blocks and 3 log segments
I20260812 06:37:44.624197 17191 ts_tablet_manager.cc:1916] T c0b0a910c55b4f928acd0a5e4b346673 P d41d668ab84c421381071558fe019747: Deleting tablet data with delete state TABLET_DATA_COPYING
I20260812 06:37:44.626250 17191 ts_tablet_manager.cc:1929] T c0b0a910c55b4f928acd0a5e4b346673 P d41d668ab84c421381071558fe019747: tablet deleted with delete type TABLET_DATA_COPYING: last-logged OpId 1.2
I20260812 06:37:44.626504 17191 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c0b0a910c55b4f928acd0a5e4b346673. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 06:37:44.627763 17191 tablet_copy_client.cc:806] T c0b0a910c55b4f928acd0a5e4b346673 P d41d668ab84c421381071558fe019747: tablet copy: Starting download of 5 data blocks...
I20260812 06:37:44.635043 17191 tablet_copy_client.cc:670] T c0b0a910c55b4f928acd0a5e4b346673 P d41d668ab84c421381071558fe019747: tablet copy: Starting download of 3 WAL segments...
I20260812 06:37:44.644398 17191 tablet_copy_client.cc:538] T c0b0a910c55b4f928acd0a5e4b346673 P d41d668ab84c421381071558fe019747: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260812 06:37:44.646644 17191 tablet_bootstrap.cc:492] T c0b0a910c55b4f928acd0a5e4b346673 P d41d668ab84c421381071558fe019747: Bootstrap starting.
I20260812 06:37:44.657543 17191 tablet_bootstrap.cc:492] T c0b0a910c55b4f928acd0a5e4b346673 P d41d668ab84c421381071558fe019747: Bootstrap replayed 1/3 log segments. Stats: ops{read=7 overwritten=0 applied=5 ignored=5} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=2. Pending: 2 replicates
I20260812 06:37:44.661942 17191 tablet_bootstrap.cc:492] T c0b0a910c55b4f928acd0a5e4b346673 P d41d668ab84c421381071558fe019747: Bootstrap replayed 2/3 log segments. Stats: ops{read=14 overwritten=0 applied=12 ignored=12} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=2. Pending: 2 replicates
I20260812 06:37:44.666612 17191 tablet_bootstrap.cc:492] T c0b0a910c55b4f928acd0a5e4b346673 P d41d668ab84c421381071558fe019747: Bootstrap replayed 3/3 log segments. Stats: ops{read=22 overwritten=0 applied=22 ignored=20} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=2. Pending: 0 replicates
I20260812 06:37:44.666975 17191 tablet_bootstrap.cc:492] T c0b0a910c55b4f928acd0a5e4b346673 P d41d668ab84c421381071558fe019747: Bootstrap complete.
I20260812 06:37:44.667677 17191 ts_tablet_manager.cc:1403] T c0b0a910c55b4f928acd0a5e4b346673 P d41d668ab84c421381071558fe019747: Time spent bootstrapping tablet: real 0.021s	user 0.015s	sys 0.004s
I20260812 06:37:44.668229 17191 raft_consensus.cc:359] T c0b0a910c55b4f928acd0a5e4b346673 P d41d668ab84c421381071558fe019747 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 44 OBSOLETE_local: false peers { permanent_uuid: "68b666b388e54d3a93ae86c6098c7b44" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 40995 } } peers { permanent_uuid: "d42658cb732d4733b992c5e7a7540576" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 38765 } } peers { permanent_uuid: "d41d668ab84c421381071558fe019747" member_type: NON_VOTER last_known_addr { host: "127.4.177.3" port: 36879 } attrs { promote: true } }
I20260812 06:37:44.668335 17191 raft_consensus.cc:740] T c0b0a910c55b4f928acd0a5e4b346673 P d41d668ab84c421381071558fe019747 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: d41d668ab84c421381071558fe019747, State: Initialized, Role: LEARNER
I20260812 06:37:44.668493 17191 consensus_queue.cc:260] T c0b0a910c55b4f928acd0a5e4b346673 P d41d668ab84c421381071558fe019747 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 44, Last appended: 1.44, Last appended by leader: 44, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 44 OBSOLETE_local: false peers { permanent_uuid: "68b666b388e54d3a93ae86c6098c7b44" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 40995 } } peers { permanent_uuid: "d42658cb732d4733b992c5e7a7540576" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 38765 } } peers { permanent_uuid: "d41d668ab84c421381071558fe019747" member_type: NON_VOTER last_known_addr { host: "127.4.177.3" port: 36879 } attrs { promote: true } }
I20260812 06:37:44.668867 17191 ts_tablet_manager.cc:1434] T c0b0a910c55b4f928acd0a5e4b346673 P d41d668ab84c421381071558fe019747: Time spent starting tablet: real 0.001s	user 0.002s	sys 0.000s
I20260812 06:37:44.669713 16773 tablet_copy_service.cc:342] P d42658cb732d4733b992c5e7a7540576: Request end of tablet copy session d41d668ab84c421381071558fe019747-c0b0a910c55b4f928acd0a5e4b346673 received from {username='slave'} at 127.4.177.3:53301
I20260812 06:37:44.669811 16773 tablet_copy_service.cc:434] P d42658cb732d4733b992c5e7a7540576: ending tablet copy session d41d668ab84c421381071558fe019747-c0b0a910c55b4f928acd0a5e4b346673 on tablet c0b0a910c55b4f928acd0a5e4b346673 with peer d41d668ab84c421381071558fe019747
I20260812 06:37:44.755404 17053 maintenance_manager.cc:419] P d41d668ab84c421381071558fe019747: Scheduling UndoDeltaBlockGCOp(c0b0a910c55b4f928acd0a5e4b346673): 595 bytes on disk
I20260812 06:37:44.756644 16859 maintenance_manager.cc:643] P d41d668ab84c421381071558fe019747: UndoDeltaBlockGCOp(c0b0a910c55b4f928acd0a5e4b346673) complete. Timing: real 0.001s	user 0.000s	sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":72,"lbm_reads_lt_1ms":4}
I20260812 06:37:45.161088 16963 raft_consensus.cc:1217] T c0b0a910c55b4f928acd0a5e4b346673 P d41d668ab84c421381071558fe019747 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.43->[1.44-1.44]   Dedup: 1.44->[]
I20260812 06:37:45.256402 16828 maintenance_manager.cc:419] P d42658cb732d4733b992c5e7a7540576: Scheduling LogGCOp(c0b0a910c55b4f928acd0a5e4b346673): free 1840912 bytes of WAL
I20260812 06:37:45.256795 16619 log_reader.cc:385] T c0b0a910c55b4f928acd0a5e4b346673: removed 2 log segments from log reader
I20260812 06:37:45.256857 16619 log.cc:1079] T c0b0a910c55b4f928acd0a5e4b346673 P d42658cb732d4733b992c5e7a7540576: Deleting log segment in path: /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.EncryptionEnabled_RaftConsensusParamEncryptionITest.TestFollowerFallsBehindLeaderGC_0.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-1/wal/wals/c0b0a910c55b4f928acd0a5e4b346673/wal-000000004 (ops 23-29)
I20260812 06:37:45.256959 16619 log.cc:1079] T c0b0a910c55b4f928acd0a5e4b346673 P d42658cb732d4733b992c5e7a7540576: Deleting log segment in path: /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.EncryptionEnabled_RaftConsensusParamEncryptionITest.TestFollowerFallsBehindLeaderGC_0.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-1/wal/wals/c0b0a910c55b4f928acd0a5e4b346673/wal-000000005 (ops 30-36)
I20260812 06:37:45.257606 16619 maintenance_manager.cc:643] P d42658cb732d4733b992c5e7a7540576: LogGCOp(c0b0a910c55b4f928acd0a5e4b346673) complete. Timing: real 0.001s	user 0.000s	sys 0.000s Metrics: {}
I20260812 06:37:45.467866 16585 maintenance_manager.cc:419] P 68b666b388e54d3a93ae86c6098c7b44: Scheduling LogGCOp(c0b0a910c55b4f928acd0a5e4b346673): free 1840740 bytes of WAL
I20260812 06:37:45.468189 16396 log_reader.cc:385] T c0b0a910c55b4f928acd0a5e4b346673: removed 2 log segments from log reader
I20260812 06:37:45.468274 16396 log.cc:1079] T c0b0a910c55b4f928acd0a5e4b346673 P 68b666b388e54d3a93ae86c6098c7b44: Deleting log segment in path: /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.EncryptionEnabled_RaftConsensusParamEncryptionITest.TestFollowerFallsBehindLeaderGC_0.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-0/wal/wals/c0b0a910c55b4f928acd0a5e4b346673/wal-000000004 (ops 23-29)
I20260812 06:37:45.468360 16396 log.cc:1079] T c0b0a910c55b4f928acd0a5e4b346673 P 68b666b388e54d3a93ae86c6098c7b44: Deleting log segment in path: /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.EncryptionEnabled_RaftConsensusParamEncryptionITest.TestFollowerFallsBehindLeaderGC_0.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-0/wal/wals/c0b0a910c55b4f928acd0a5e4b346673/wal-000000005 (ops 30-36)
I20260812 06:37:45.469063 16396 maintenance_manager.cc:643] P 68b666b388e54d3a93ae86c6098c7b44: LogGCOp(c0b0a910c55b4f928acd0a5e4b346673) complete. Timing: real 0.001s	user 0.000s	sys 0.000s Metrics: {}
I20260812 06:37:45.672766 17206 raft_consensus.cc:1064] T c0b0a910c55b4f928acd0a5e4b346673 P d42658cb732d4733b992c5e7a7540576: attempting to promote NON_VOTER d41d668ab84c421381071558fe019747 to VOTER
I20260812 06:37:45.673264 17206 consensus_queue.cc:237] T c0b0a910c55b4f928acd0a5e4b346673 P d42658cb732d4733b992c5e7a7540576 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 44, Committed index: 44, Last appended: 1.44, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 45 OBSOLETE_local: false peers { permanent_uuid: "68b666b388e54d3a93ae86c6098c7b44" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 40995 } } peers { permanent_uuid: "d42658cb732d4733b992c5e7a7540576" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 38765 } } peers { permanent_uuid: "d41d668ab84c421381071558fe019747" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 36879 } attrs { promote: false } }
I20260812 06:37:45.674662 16963 raft_consensus.cc:1275] T c0b0a910c55b4f928acd0a5e4b346673 P d41d668ab84c421381071558fe019747 [term 1 LEARNER]: Refusing update from remote peer d42658cb732d4733b992c5e7a7540576: Log matching property violated. Preceding OpId in replica: term: 1 index: 44. Preceding OpId from leader: term: 1 index: 45. (index mismatch)
I20260812 06:37:45.674765 16501 raft_consensus.cc:1275] T c0b0a910c55b4f928acd0a5e4b346673 P 68b666b388e54d3a93ae86c6098c7b44 [term 1 FOLLOWER]: Refusing update from remote peer d42658cb732d4733b992c5e7a7540576: Log matching property violated. Preceding OpId in replica: term: 1 index: 44. Preceding OpId from leader: term: 1 index: 45. (index mismatch)
I20260812 06:37:45.675117 17210 consensus_queue.cc:1048] T c0b0a910c55b4f928acd0a5e4b346673 P d42658cb732d4733b992c5e7a7540576 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d41d668ab84c421381071558fe019747" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 36879 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 45, Last known committed idx: 44, Time since last communication: 0.000s
I20260812 06:37:45.675323 17206 consensus_queue.cc:1048] T c0b0a910c55b4f928acd0a5e4b346673 P d42658cb732d4733b992c5e7a7540576 [LEADER]: Connected to new peer: Peer: permanent_uuid: "68b666b388e54d3a93ae86c6098c7b44" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 40995 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 45, Last known committed idx: 44, Time since last communication: 0.000s
I20260812 06:37:45.677341 17205 raft_consensus.cc:2955] T c0b0a910c55b4f928acd0a5e4b346673 P d42658cb732d4733b992c5e7a7540576 [term 1 LEADER]: Committing config change with OpId 1.45: config changed from index 44 to 45, d41d668ab84c421381071558fe019747 (127.4.177.3) changed from NON_VOTER to VOTER. New config: { opid_index: 45 OBSOLETE_local: false peers { permanent_uuid: "68b666b388e54d3a93ae86c6098c7b44" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 40995 } } peers { permanent_uuid: "d42658cb732d4733b992c5e7a7540576" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 38765 } } peers { permanent_uuid: "d41d668ab84c421381071558fe019747" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 36879 } attrs { promote: false } } }
I20260812 06:37:45.678015 16501 raft_consensus.cc:2955] T c0b0a910c55b4f928acd0a5e4b346673 P 68b666b388e54d3a93ae86c6098c7b44 [term 1 FOLLOWER]: Committing config change with OpId 1.45: config changed from index 44 to 45, d41d668ab84c421381071558fe019747 (127.4.177.3) changed from NON_VOTER to VOTER. New config: { opid_index: 45 OBSOLETE_local: false peers { permanent_uuid: "68b666b388e54d3a93ae86c6098c7b44" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 40995 } } peers { permanent_uuid: "d42658cb732d4733b992c5e7a7540576" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 38765 } } peers { permanent_uuid: "d41d668ab84c421381071558fe019747" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 36879 } attrs { promote: false } } }
I20260812 06:37:45.677904 16963 raft_consensus.cc:2955] T c0b0a910c55b4f928acd0a5e4b346673 P d41d668ab84c421381071558fe019747 [term 1 FOLLOWER]: Committing config change with OpId 1.45: config changed from index 44 to 45, d41d668ab84c421381071558fe019747 (127.4.177.3) changed from NON_VOTER to VOTER. New config: { opid_index: 45 OBSOLETE_local: false peers { permanent_uuid: "68b666b388e54d3a93ae86c6098c7b44" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 40995 } } peers { permanent_uuid: "d42658cb732d4733b992c5e7a7540576" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 38765 } } peers { permanent_uuid: "d41d668ab84c421381071558fe019747" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 36879 } attrs { promote: false } } }
I20260812 06:37:45.680126 16265 catalog_manager.cc:5719] T c0b0a910c55b4f928acd0a5e4b346673 P d42658cb732d4733b992c5e7a7540576 reported cstate change: config changed from index 44 to 45, d41d668ab84c421381071558fe019747 (127.4.177.3) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "d42658cb732d4733b992c5e7a7540576" committed_config { opid_index: 45 OBSOLETE_local: false peers { permanent_uuid: "68b666b388e54d3a93ae86c6098c7b44" member_type: VOTER last_known_addr { host: "127.4.177.1" port: 40995 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d42658cb732d4733b992c5e7a7540576" member_type: VOTER last_known_addr { host: "127.4.177.2" port: 38765 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d41d668ab84c421381071558fe019747" member_type: VOTER last_known_addr { host: "127.4.177.3" port: 36879 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260812 06:37:45.760576 17053 maintenance_manager.cc:419] P d41d668ab84c421381071558fe019747: Scheduling LogGCOp(c0b0a910c55b4f928acd0a5e4b346673): free 1840860 bytes of WAL
I20260812 06:37:45.760902 16859 log_reader.cc:385] T c0b0a910c55b4f928acd0a5e4b346673: removed 2 log segments from log reader
I20260812 06:37:45.760984 16859 log.cc:1079] T c0b0a910c55b4f928acd0a5e4b346673 P d41d668ab84c421381071558fe019747: Deleting log segment in path: /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.EncryptionEnabled_RaftConsensusParamEncryptionITest.TestFollowerFallsBehindLeaderGC_0.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-2/wal/wals/c0b0a910c55b4f928acd0a5e4b346673/wal-000000001 (ops 23-29)
I20260812 06:37:45.761024 16859 log.cc:1079] T c0b0a910c55b4f928acd0a5e4b346673 P d41d668ab84c421381071558fe019747: Deleting log segment in path: /tmp/dist-test-taskCnZu03/test-tmp/raft_consensus-itest.2.EncryptionEnabled_RaftConsensusParamEncryptionITest.TestFollowerFallsBehindLeaderGC_0.1786516568546459-4804-0/raft_consensus-itest-cluster/ts-2/wal/wals/c0b0a910c55b4f928acd0a5e4b346673/wal-000000002 (ops 30-36)
I20260812 06:37:45.761567 16859 maintenance_manager.cc:643] P d41d668ab84c421381071558fe019747: LogGCOp(c0b0a910c55b4f928acd0a5e4b346673) complete. Timing: real 0.001s	user 0.000s	sys 0.000s Metrics: {}
I20260812 06:37:48.468122  4804 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskCnZu03/build/debug/bin/kudu with pid 16365
I20260812 06:37:48.486660  4804 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskCnZu03/build/debug/bin/kudu with pid 16588
I20260812 06:37:48.497756  4804 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskCnZu03/build/debug/bin/kudu with pid 16831
I20260812 06:37:48.513345  4804 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskCnZu03/build/debug/bin/kudu with pid 16212
2026-08-12T06:37:48Z chronyd exiting
[       OK ] EncryptionEnabled/RaftConsensusParamEncryptionITest.TestFollowerFallsBehindLeaderGC/0 (9069 ms)
[----------] 1 test from EncryptionEnabled/RaftConsensusParamEncryptionITest (9069 ms total)

[----------] Global test environment tear-down
[==========] 7 tests from 2 test suites ran. (99988 ms total)
[  PASSED  ] 7 tests.
I20260812 06:37:48.547029  4804 logging.cc:436] LogThrottler /workspace/apache/dev/local/kudu/src/kudu/client/batcher.cc:441: suppressed but not reported on 11 messages since previous log ~15 seconds ago
