[==========] Running 17 tests from 3 test suites.
[----------] Global test environment set-up.
[----------] 2 tests from MockHybridClockTest
[ RUN      ] MockHybridClockTest.TestMockedSystemClock
WARNING: Logging before InitGoogleLogging() is written to STDERR
I20260812 08:03:47.837185 22218 hybrid_clock.cc:584] initializing the hybrid clock with 'mock' time source
I20260812 08:03:47.837347 22218 hybrid_clock.cc:648] HybridClock initialized: now 0 us; error 0 us; skew 500 ppm
[       OK ] MockHybridClockTest.TestMockedSystemClock (2 ms)
[ RUN      ] MockHybridClockTest.TestClockDealsWithWrapping
I20260812 08:03:47.838857 22218 hybrid_clock.cc:584] initializing the hybrid clock with 'mock' time source
I20260812 08:03:47.838915 22218 hybrid_clock.cc:648] HybridClock initialized: now 0 us; error 0 us; skew 500 ppm
[       OK ] MockHybridClockTest.TestClockDealsWithWrapping (9 ms)
[----------] 2 tests from MockHybridClockTest (12 ms total)

[----------] 13 tests from HybridClockTest
[ RUN      ] HybridClockTest.NowValuesIncreaseMonotonically
I20260812 08:03:47.848347 22218 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260812 08:03:47.848402 22218 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260812 08:03:47.848456 22218 hybrid_clock.cc:648] HybridClock initialized: now 1786521827848455 us; error 0 us; skew 500 ppm
[       OK ] HybridClockTest.NowValuesIncreaseMonotonically (1 ms)
[ RUN      ] HybridClockTest.UpdateLogicalValueIncreasesByAmount
I20260812 08:03:47.849337 22218 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260812 08:03:47.849377 22218 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260812 08:03:47.849392 22218 hybrid_clock.cc:648] HybridClock initialized: now 1786521827849392 us; error 0 us; skew 500 ppm
[       OK ] HybridClockTest.UpdateLogicalValueIncreasesByAmount (0 ms)
[ RUN      ] HybridClockTest.WaitUntilAfterCase1
I20260812 08:03:47.850183 22218 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260812 08:03:47.850220 22218 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260812 08:03:47.850234 22218 hybrid_clock.cc:648] HybridClock initialized: now 1786521827850235 us; error 0 us; skew 500 ppm
[       OK ] HybridClockTest.WaitUntilAfterCase1 (0 ms)
[ RUN      ] HybridClockTest.WaitUntilAfterCase2
I20260812 08:03:47.851064 22218 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260812 08:03:47.851102 22218 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260812 08:03:47.851116 22218 hybrid_clock.cc:648] HybridClock initialized: now 1786521827851117 us; error 0 us; skew 500 ppm
[       OK ] HybridClockTest.WaitUntilAfterCase2 (2 ms)
[ RUN      ] HybridClockTest.TestIsAfter
I20260812 08:03:47.854101 22218 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260812 08:03:47.854140 22218 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260812 08:03:47.854156 22218 hybrid_clock.cc:648] HybridClock initialized: now 1786521827854155 us; error 0 us; skew 500 ppm
[       OK ] HybridClockTest.TestIsAfter (0 ms)
[ RUN      ] HybridClockTest.TestClockDoesntGoBackwardsWithUpdates
I20260812 08:03:47.854952 22218 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260812 08:03:47.854990 22218 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260812 08:03:47.855003 22218 hybrid_clock.cc:648] HybridClock initialized: now 1786521827855004 us; error 0 us; skew 500 ppm
[       OK ] HybridClockTest.TestClockDoesntGoBackwardsWithUpdates (1001 ms)
[ RUN      ] HybridClockTest.TestGetPhysicalComponentDifference
I20260812 08:03:48.856972 22218 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260812 08:03:48.857028 22218 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260812 08:03:48.857044 22218 hybrid_clock.cc:648] HybridClock initialized: now 1786521828857045 us; error 0 us; skew 500 ppm
[       OK ] HybridClockTest.TestGetPhysicalComponentDifference (2 ms)
[ RUN      ] HybridClockTest.TestRideOverNtpInterruption
I20260812 08:03:48.858956 22218 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260812 08:03:48.858999 22218 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260812 08:03:48.859014 22218 hybrid_clock.cc:648] HybridClock initialized: now 1786521828859013 us; error 0 us; skew 500 ppm
E20260812 08:03:49.859264 22218 hybrid_clock.cc:750] unable to read clock for last 1.000s: Service unavailable: Injected clock unsync error
[       OK ] HybridClockTest.TestRideOverNtpInterruption (1001 ms)
[ RUN      ] HybridClockTest.SlowClockInitialisation
I20260812 08:03:49.860682 22218 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260812 08:03:49.860739 22218 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260812 08:03:49.860757 22218 hybrid_clock.cc:648] HybridClock initialized: now 1786521829860757 us; error 0 us; skew 500 ppm
2026-08-12T08:03:49Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2026-08-12T08:03:49Z Disabled control of system clock
I20260812 08:03:49.878414 22218 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskijWYiT/build/debug/bin/kudu
/tmp/dist-test-taskijWYiT/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskijWYiT/test-tmp/hybrid_clock-test.0.HybridClockTest.SlowClockInitialisation.1786521827826957-22218-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskijWYiT/test-tmp/hybrid_clock-test.0.HybridClockTest.SlowClockInitialisation.1786521827826957-22218-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskijWYiT/test-tmp/hybrid_clock-test.0.HybridClockTest.SlowClockInitialisation.1786521827826957-22218-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskijWYiT/test-tmp/hybrid_clock-test.0.HybridClockTest.SlowClockInitialisation.1786521827826957-22218-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.178.190:34091
--webserver_interface=127.21.178.190
--webserver_port=0
--builtin_ntp_servers=127.21.178.148:45861
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.178.190:34091,127.21.178.189:33967,127.21.178.188:36361
--hybrid_clock_inject_init_delay_ms=100
--wall_clock_jump_detection=enabled with env {}
W20260812 08:03:49.997542 22232 flags.cc:432] Enabled unsafe flag: --hybrid_clock_inject_init_delay_ms=100
W20260812 08:03:49.997879 22232 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 08:03:49.997951 22232 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 08:03:49.998051 22232 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 08:03:50.001574 22232 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 08:03:50.001678 22232 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 08:03:50.001716 22232 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 08:03:50.001755 22232 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
W20260812 08:03:50.001801 22232 flags.cc:432] Enabled experimental flag: --wall_clock_jump_detection=enabled
I20260812 08:03:50.006310 22232 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskijWYiT/test-tmp/hybrid_clock-test.0.HybridClockTest.SlowClockInitialisation.1786521827826957-22218-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.178.148:45861
--hybrid_clock_inject_init_delay_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskijWYiT/test-tmp/hybrid_clock-test.0.HybridClockTest.SlowClockInitialisation.1786521827826957-22218-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskijWYiT/test-tmp/hybrid_clock-test.0.HybridClockTest.SlowClockInitialisation.1786521827826957-22218-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.178.190:34091,127.21.178.189:33967,127.21.178.188:36361
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.178.190:34091
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--wall_clock_jump_detection=enabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskijWYiT/test-tmp/hybrid_clock-test.0.HybridClockTest.SlowClockInitialisation.1786521827826957-22218-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.178.190
--webserver_port=0
--never_fsync=true
--heap_profile_path=/tmp/kudu.22232
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false

Master server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 08:03:50.007593 22232 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 08:03:50.008869 22232 file_cache.cc:504] Constructed file cache file cache with capacity 419430
I20260812 08:03:50.012316 22232 server_base.cc:839] enabling wall clock jump detection
I20260812 08:03:50.012714 22232 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
W20260812 08:03:50.012892 22232 hybrid_clock.cc:606] Injecting 100ms delay in HybridClock initialization process
I20260812 08:03:50.114602 22232 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 08:03:50.115842 22232 hybrid_clock.cc:648] HybridClock initialized: now 1786521830115783 us; error 91 us; skew 500 ppm
I20260812 08:03:50.118176 22232 webserver.cc:533] Webserver started at http://127.21.178.190:39985/ using document root <none> and password file <none>
I20260812 08:03:50.118841 22232 fs_manager.cc:362] Metadata directory not provided
I20260812 08:03:50.118942 22232 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 08:03:50.119180 22232 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 08:03:50.120899 22232 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskijWYiT/test-tmp/hybrid_clock-test.0.HybridClockTest.SlowClockInitialisation.1786521827826957-22218-0/minicluster-data/master-0/data/instance:
uuid: "43d0f953cc1d438db131bdf6e1ac6c78"
format_stamp: "Formatted at 2026-08-12 08:03:50 on dist-test-slave-8w9v"
I20260812 08:03:50.121433 22232 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskijWYiT/test-tmp/hybrid_clock-test.0.HybridClockTest.SlowClockInitialisation.1786521827826957-22218-0/minicluster-data/master-0/wal/instance:
uuid: "43d0f953cc1d438db131bdf6e1ac6c78"
format_stamp: "Formatted at 2026-08-12 08:03:50 on dist-test-slave-8w9v"
I20260812 08:03:50.125154 22232 fs_manager.cc:696] Time spent creating directory manager: real 0.003s	user 0.004s	sys 0.000s
I20260812 08:03:50.127363 22242 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:03:50.128427 22232 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.001s
I20260812 08:03:50.128566 22232 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskijWYiT/test-tmp/hybrid_clock-test.0.HybridClockTest.SlowClockInitialisation.1786521827826957-22218-0/minicluster-data/master-0/data,/tmp/dist-test-taskijWYiT/test-tmp/hybrid_clock-test.0.HybridClockTest.SlowClockInitialisation.1786521827826957-22218-0/minicluster-data/master-0/wal
uuid: "43d0f953cc1d438db131bdf6e1ac6c78"
format_stamp: "Formatted at 2026-08-12 08:03:50 on dist-test-slave-8w9v"
I20260812 08:03:50.128671 22232 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskijWYiT/test-tmp/hybrid_clock-test.0.HybridClockTest.SlowClockInitialisation.1786521827826957-22218-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskijWYiT/test-tmp/hybrid_clock-test.0.HybridClockTest.SlowClockInitialisation.1786521827826957-22218-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskijWYiT/test-tmp/hybrid_clock-test.0.HybridClockTest.SlowClockInitialisation.1786521827826957-22218-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 08:03:50.149850 22232 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:03:50.150638 22232 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 08:03:50.150857 22232 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:03:50.158967 22299 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.21.178.190:34091 every 8 connection(s)
I20260812 08:03:50.158969 22232 rpc_server.cc:307] RPC server started. Bound to: 127.21.178.190:34091
I20260812 08:03:50.160153 22232 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskijWYiT/test-tmp/hybrid_clock-test.0.HybridClockTest.SlowClockInitialisation.1786521827826957-22218-0/minicluster-data/master-0/data/info.pb
I20260812 08:03:50.163242 22300 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 08:03:50.166488 22300 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.178.190" port: 34091 } has no permanent_uuid. Determining permanent_uuid...
I20260812 08:03:50.167297 22218 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskijWYiT/build/debug/bin/kudu as pid 22232
I20260812 08:03:50.167445 22218 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskijWYiT/test-tmp/hybrid_clock-test.0.HybridClockTest.SlowClockInitialisation.1786521827826957-22218-0/minicluster-data/master-0/wal/instance
I20260812 08:03:50.168890 22218 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskijWYiT/build/debug/bin/kudu
/tmp/dist-test-taskijWYiT/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskijWYiT/test-tmp/hybrid_clock-test.0.HybridClockTest.SlowClockInitialisation.1786521827826957-22218-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskijWYiT/test-tmp/hybrid_clock-test.0.HybridClockTest.SlowClockInitialisation.1786521827826957-22218-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskijWYiT/test-tmp/hybrid_clock-test.0.HybridClockTest.SlowClockInitialisation.1786521827826957-22218-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskijWYiT/test-tmp/hybrid_clock-test.0.HybridClockTest.SlowClockInitialisation.1786521827826957-22218-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.178.189:33967
--webserver_interface=127.21.178.189
--webserver_port=0
--builtin_ntp_servers=127.21.178.148:45861
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.178.190:34091,127.21.178.189:33967,127.21.178.188:36361
--hybrid_clock_inject_init_delay_ms=100
--wall_clock_jump_detection=enabled with env {}
I20260812 08:03:50.189549 22300 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.178.189" port: 33967 } has no permanent_uuid. Determining permanent_uuid...
W20260812 08:03:50.190550 22243 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.21.178.189:33967: connect: Connection refused (error 111)
W20260812 08:03:50.191569 22300 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.21.178.189:33967: Network error: Client connection negotiation failed: client connection to 127.21.178.189:33967: connect: Connection refused (error 111)
I20260812 08:03:50.240861 22300 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.178.189" port: 33967 } attempt: 1
W20260812 08:03:50.242666 22300 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.21.178.189:33967: Network error: Client connection negotiation failed: client connection to 127.21.178.189:33967: connect: Connection refused (error 111)
W20260812 08:03:50.287671 22305 flags.cc:432] Enabled unsafe flag: --hybrid_clock_inject_init_delay_ms=100
W20260812 08:03:50.287961 22305 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 08:03:50.287995 22305 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 08:03:50.288038 22305 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 08:03:50.291465 22305 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 08:03:50.291553 22305 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 08:03:50.291576 22305 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 08:03:50.291597 22305 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
W20260812 08:03:50.291620 22305 flags.cc:432] Enabled experimental flag: --wall_clock_jump_detection=enabled
I20260812 08:03:50.295665 22305 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskijWYiT/test-tmp/hybrid_clock-test.0.HybridClockTest.SlowClockInitialisation.1786521827826957-22218-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.178.148:45861
--hybrid_clock_inject_init_delay_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskijWYiT/test-tmp/hybrid_clock-test.0.HybridClockTest.SlowClockInitialisation.1786521827826957-22218-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskijWYiT/test-tmp/hybrid_clock-test.0.HybridClockTest.SlowClockInitialisation.1786521827826957-22218-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.21.178.190:34091,127.21.178.189:33967,127.21.178.188:36361
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.178.189:33967
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--wall_clock_jump_detection=enabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskijWYiT/test-tmp/hybrid_clock-test.0.HybridClockTest.SlowClockInitialisation.1786521827826957-22218-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.21.178.189
--webserver_port=0
--never_fsync=true
--heap_profile_path=/tmp/kudu.22305
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false

Master server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 08:03:50.296880 22305 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 08:03:50.298022 22305 file_cache.cc:504] Constructed file cache file cache with capacity 419430
I20260812 08:03:50.301177 22305 server_base.cc:839] enabling wall clock jump detection
I20260812 08:03:50.301538 22305 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
W20260812 08:03:50.301676 22305 hybrid_clock.cc:606] Injecting 100ms delay in HybridClock initialization process
I20260812 08:03:50.310981 22300 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.178.189" port: 33967 } attempt: 2
W20260812 08:03:50.312619 22300 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.21.178.189:33967: Network error: Client connection negotiation failed: client connection to 127.21.178.189:33967: connect: Connection refused (error 111)
I20260812 08:03:50.403388 22305 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 08:03:50.403784 22300 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.178.189" port: 33967 } attempt: 3
I20260812 08:03:50.404629 22305 hybrid_clock.cc:648] HybridClock initialized: now 1786521830404610 us; error 46 us; skew 500 ppm
W20260812 08:03:50.405468 22300 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.21.178.189:33967: Network error: Client connection negotiation failed: client connection to 127.21.178.189:33967: connect: Connection refused (error 111)
I20260812 08:03:50.407130 22305 webserver.cc:533] Webserver started at http://127.21.178.189:44687/ using document root <none> and password file <none>
I20260812 08:03:50.407783 22305 fs_manager.cc:362] Metadata directory not provided
I20260812 08:03:50.407869 22305 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 08:03:50.408097 22305 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 08:03:50.409828 22305 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskijWYiT/test-tmp/hybrid_clock-test.0.HybridClockTest.SlowClockInitialisation.1786521827826957-22218-0/minicluster-data/master-1/data/instance:
uuid: "6d7aeaee8b1c46d29a54da6cc92cbf3d"
format_stamp: "Formatted at 2026-08-12 08:03:50 on dist-test-slave-8w9v"
I20260812 08:03:50.410342 22305 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskijWYiT/test-tmp/hybrid_clock-test.0.HybridClockTest.SlowClockInitialisation.1786521827826957-22218-0/minicluster-data/master-1/wal/instance:
uuid: "6d7aeaee8b1c46d29a54da6cc92cbf3d"
format_stamp: "Formatted at 2026-08-12 08:03:50 on dist-test-slave-8w9v"
I20260812 08:03:50.414114 22305 fs_manager.cc:696] Time spent creating directory manager: real 0.003s	user 0.003s	sys 0.002s
I20260812 08:03:50.416723 22316 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:03:50.417871 22305 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260812 08:03:50.418035 22305 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskijWYiT/test-tmp/hybrid_clock-test.0.HybridClockTest.SlowClockInitialisation.1786521827826957-22218-0/minicluster-data/master-1/data,/tmp/dist-test-taskijWYiT/test-tmp/hybrid_clock-test.0.HybridClockTest.SlowClockInitialisation.1786521827826957-22218-0/minicluster-data/master-1/wal
uuid: "6d7aeaee8b1c46d29a54da6cc92cbf3d"
format_stamp: "Formatted at 2026-08-12 08:03:50 on dist-test-slave-8w9v"
I20260812 08:03:50.418162 22305 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskijWYiT/test-tmp/hybrid_clock-test.0.HybridClockTest.SlowClockInitialisation.1786521827826957-22218-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskijWYiT/test-tmp/hybrid_clock-test.0.HybridClockTest.SlowClockInitialisation.1786521827826957-22218-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskijWYiT/test-tmp/hybrid_clock-test.0.HybridClockTest.SlowClockInitialisation.1786521827826957-22218-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 08:03:50.444366 22305 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:03:50.445194 22305 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 08:03:50.445428 22305 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:03:50.453781 22305 rpc_server.cc:307] RPC server started. Bound to: 127.21.178.189:33967
I20260812 08:03:50.453788 22379 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.21.178.189:33967 every 8 connection(s)
I20260812 08:03:50.454931 22305 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskijWYiT/test-tmp/hybrid_clock-test.0.HybridClockTest.SlowClockInitialisation.1786521827826957-22218-0/minicluster-data/master-1/data/info.pb
I20260812 08:03:50.457901 22380 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 08:03:50.458678 22218 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskijWYiT/build/debug/bin/kudu as pid 22305
I20260812 08:03:50.458791 22218 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskijWYiT/test-tmp/hybrid_clock-test.0.HybridClockTest.SlowClockInitialisation.1786521827826957-22218-0/minicluster-data/master-1/wal/instance
I20260812 08:03:50.459527 22218 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskijWYiT/build/debug/bin/kudu
/tmp/dist-test-taskijWYiT/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskijWYiT/test-tmp/hybrid_clock-test.0.HybridClockTest.SlowClockInitialisation.1786521827826957-22218-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskijWYiT/test-tmp/hybrid_clock-test.0.HybridClockTest.SlowClockInitialisation.1786521827826957-22218-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskijWYiT/test-tmp/hybrid_clock-test.0.HybridClockTest.SlowClockInitialisation.1786521827826957-22218-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskijWYiT/test-tmp/hybrid_clock-test.0.HybridClockTest.SlowClockInitialisation.1786521827826957-22218-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.178.188:36361
--webserver_interface=127.21.178.188
--webserver_port=0
--builtin_ntp_servers=127.21.178.148:45861
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.178.190:34091,127.21.178.189:33967,127.21.178.188:36361
--hybrid_clock_inject_init_delay_ms=100
--wall_clock_jump_detection=enabled with env {}
I20260812 08:03:50.462131 22380 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.178.190" port: 34091 } has no permanent_uuid. Determining permanent_uuid...
I20260812 08:03:50.473094 22380 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.178.189" port: 33967 } has no permanent_uuid. Determining permanent_uuid...
I20260812 08:03:50.480011 22380 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.178.188" port: 36361 } has no permanent_uuid. Determining permanent_uuid...
W20260812 08:03:50.480933 22318 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.21.178.188:36361: connect: Connection refused (error 111)
W20260812 08:03:50.481945 22380 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.21.178.188:36361: Network error: Client connection negotiation failed: client connection to 127.21.178.188:36361: connect: Connection refused (error 111)
I20260812 08:03:50.531224 22380 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.178.188" port: 36361 } attempt: 1
W20260812 08:03:50.533005 22380 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.21.178.188:36361: Network error: Client connection negotiation failed: client connection to 127.21.178.188:36361: connect: Connection refused (error 111)
I20260812 08:03:50.548666 22300 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.178.189" port: 33967 } attempt: 4
I20260812 08:03:50.553903 22300 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.178.188" port: 36361 } has no permanent_uuid. Determining permanent_uuid...
W20260812 08:03:50.555452 22300 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.21.178.188:36361: Network error: Client connection negotiation failed: client connection to 127.21.178.188:36361: connect: Connection refused (error 111)
W20260812 08:03:50.574636 22384 flags.cc:432] Enabled unsafe flag: --hybrid_clock_inject_init_delay_ms=100
W20260812 08:03:50.574926 22384 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 08:03:50.574990 22384 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 08:03:50.575084 22384 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 08:03:50.578724 22384 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 08:03:50.578835 22384 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 08:03:50.578860 22384 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 08:03:50.578878 22384 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
W20260812 08:03:50.578900 22384 flags.cc:432] Enabled experimental flag: --wall_clock_jump_detection=enabled
I20260812 08:03:50.583137 22384 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskijWYiT/test-tmp/hybrid_clock-test.0.HybridClockTest.SlowClockInitialisation.1786521827826957-22218-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.178.148:45861
--hybrid_clock_inject_init_delay_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskijWYiT/test-tmp/hybrid_clock-test.0.HybridClockTest.SlowClockInitialisation.1786521827826957-22218-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskijWYiT/test-tmp/hybrid_clock-test.0.HybridClockTest.SlowClockInitialisation.1786521827826957-22218-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.21.178.190:34091,127.21.178.189:33967,127.21.178.188:36361
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.178.188:36361
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--wall_clock_jump_detection=enabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskijWYiT/test-tmp/hybrid_clock-test.0.HybridClockTest.SlowClockInitialisation.1786521827826957-22218-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.21.178.188
--webserver_port=0
--never_fsync=true
--heap_profile_path=/tmp/kudu.22384
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false

Master server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 08:03:50.584501 22384 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 08:03:50.585707 22384 file_cache.cc:504] Constructed file cache file cache with capacity 419430
I20260812 08:03:50.588945 22384 server_base.cc:839] enabling wall clock jump detection
I20260812 08:03:50.589334 22384 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
W20260812 08:03:50.589514 22384 hybrid_clock.cc:606] Injecting 100ms delay in HybridClock initialization process
I20260812 08:03:50.601225 22380 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.178.188" port: 36361 } attempt: 2
W20260812 08:03:50.602778 22380 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.21.178.188:36361: Network error: Client connection negotiation failed: client connection to 127.21.178.188:36361: connect: Connection refused (error 111)
I20260812 08:03:50.614660 22300 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.178.188" port: 36361 } attempt: 1
W20260812 08:03:50.616343 22300 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.21.178.188:36361: Network error: Client connection negotiation failed: client connection to 127.21.178.188:36361: connect: Connection refused (error 111)
I20260812 08:03:50.683614 22300 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.178.188" port: 36361 } attempt: 2
W20260812 08:03:50.685474 22300 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.21.178.188:36361: Network error: Client connection negotiation failed: client connection to 127.21.178.188:36361: connect: Connection refused (error 111)
I20260812 08:03:50.691253 22384 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 08:03:50.692497 22384 hybrid_clock.cc:648] HybridClock initialized: now 1786521830692469 us; error 58 us; skew 500 ppm
I20260812 08:03:50.693984 22380 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.178.188" port: 36361 } attempt: 3
I20260812 08:03:50.694969 22384 webserver.cc:533] Webserver started at http://127.21.178.188:40415/ using document root <none> and password file <none>
W20260812 08:03:50.695781 22380 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.21.178.188:36361: Network error: Client connection negotiation failed: client connection to 127.21.178.188:36361: connect: Connection refused (error 111)
I20260812 08:03:50.695794 22384 fs_manager.cc:362] Metadata directory not provided
I20260812 08:03:50.695878 22384 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 08:03:50.696195 22384 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 08:03:50.697963 22384 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskijWYiT/test-tmp/hybrid_clock-test.0.HybridClockTest.SlowClockInitialisation.1786521827826957-22218-0/minicluster-data/master-2/data/instance:
uuid: "fe980ff120154749b98f3e31aba0cdd6"
format_stamp: "Formatted at 2026-08-12 08:03:50 on dist-test-slave-8w9v"
I20260812 08:03:50.698477 22384 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskijWYiT/test-tmp/hybrid_clock-test.0.HybridClockTest.SlowClockInitialisation.1786521827826957-22218-0/minicluster-data/master-2/wal/instance:
uuid: "fe980ff120154749b98f3e31aba0cdd6"
format_stamp: "Formatted at 2026-08-12 08:03:50 on dist-test-slave-8w9v"
I20260812 08:03:50.702119 22384 fs_manager.cc:696] Time spent creating directory manager: real 0.003s	user 0.003s	sys 0.002s
I20260812 08:03:50.704602 22398 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:03:50.705762 22384 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260812 08:03:50.705891 22384 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskijWYiT/test-tmp/hybrid_clock-test.0.HybridClockTest.SlowClockInitialisation.1786521827826957-22218-0/minicluster-data/master-2/data,/tmp/dist-test-taskijWYiT/test-tmp/hybrid_clock-test.0.HybridClockTest.SlowClockInitialisation.1786521827826957-22218-0/minicluster-data/master-2/wal
uuid: "fe980ff120154749b98f3e31aba0cdd6"
format_stamp: "Formatted at 2026-08-12 08:03:50 on dist-test-slave-8w9v"
I20260812 08:03:50.706004 22384 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskijWYiT/test-tmp/hybrid_clock-test.0.HybridClockTest.SlowClockInitialisation.1786521827826957-22218-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskijWYiT/test-tmp/hybrid_clock-test.0.HybridClockTest.SlowClockInitialisation.1786521827826957-22218-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskijWYiT/test-tmp/hybrid_clock-test.0.HybridClockTest.SlowClockInitialisation.1786521827826957-22218-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 08:03:50.723793 22384 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:03:50.724601 22384 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 08:03:50.724804 22384 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:03:50.732640 22384 rpc_server.cc:307] RPC server started. Bound to: 127.21.178.188:36361
I20260812 08:03:50.732651 22463 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.21.178.188:36361 every 8 connection(s)
I20260812 08:03:50.733748 22384 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskijWYiT/test-tmp/hybrid_clock-test.0.HybridClockTest.SlowClockInitialisation.1786521827826957-22218-0/minicluster-data/master-2/data/info.pb
I20260812 08:03:50.736753 22464 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 08:03:50.738296 22218 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskijWYiT/build/debug/bin/kudu as pid 22384
I20260812 08:03:50.738417 22218 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskijWYiT/test-tmp/hybrid_clock-test.0.HybridClockTest.SlowClockInitialisation.1786521827826957-22218-0/minicluster-data/master-2/wal/instance
I20260812 08:03:50.740046 22464 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.178.190" port: 34091 } has no permanent_uuid. Determining permanent_uuid...
I20260812 08:03:50.751222 22464 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.178.189" port: 33967 } has no permanent_uuid. Determining permanent_uuid...
I20260812 08:03:50.756212 22464 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.21.178.188" port: 36361 } has no permanent_uuid. Determining permanent_uuid...
I20260812 08:03:50.762933 22464 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fe980ff120154749b98f3e31aba0cdd6: Bootstrap starting.
I20260812 08:03:50.765373 22464 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fe980ff120154749b98f3e31aba0cdd6: Neither blocks nor log segments found. Creating new log.
I20260812 08:03:50.766155 22464 log.cc:826] T 00000000000000000000000000000000 P fe980ff120154749b98f3e31aba0cdd6: Log is configured to *not* fsync() on all Append() calls
I20260812 08:03:50.767940 22464 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fe980ff120154749b98f3e31aba0cdd6: No bootstrap required, opened a new log
I20260812 08:03:50.771517 22464 raft_consensus.cc:359] T 00000000000000000000000000000000 P fe980ff120154749b98f3e31aba0cdd6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43d0f953cc1d438db131bdf6e1ac6c78" member_type: VOTER last_known_addr { host: "127.21.178.190" port: 34091 } } peers { permanent_uuid: "6d7aeaee8b1c46d29a54da6cc92cbf3d" member_type: VOTER last_known_addr { host: "127.21.178.189" port: 33967 } } peers { permanent_uuid: "fe980ff120154749b98f3e31aba0cdd6" member_type: VOTER last_known_addr { host: "127.21.178.188" port: 36361 } }
I20260812 08:03:50.771694 22464 raft_consensus.cc:385] T 00000000000000000000000000000000 P fe980ff120154749b98f3e31aba0cdd6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 08:03:50.771761 22464 raft_consensus.cc:740] T 00000000000000000000000000000000 P fe980ff120154749b98f3e31aba0cdd6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fe980ff120154749b98f3e31aba0cdd6, State: Initialized, Role: FOLLOWER
I20260812 08:03:50.772250 22464 consensus_queue.cc:260] T 00000000000000000000000000000000 P fe980ff120154749b98f3e31aba0cdd6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43d0f953cc1d438db131bdf6e1ac6c78" member_type: VOTER last_known_addr { host: "127.21.178.190" port: 34091 } } peers { permanent_uuid: "6d7aeaee8b1c46d29a54da6cc92cbf3d" member_type: VOTER last_known_addr { host: "127.21.178.189" port: 33967 } } peers { permanent_uuid: "fe980ff120154749b98f3e31aba0cdd6" member_type: VOTER last_known_addr { host: "127.21.178.188" port: 36361 } }
I20260812 08:03:50.772745 22475 sys_catalog.cc:455] T 00000000000000000000000000000000 P fe980ff120154749b98f3e31aba0cdd6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43d0f953cc1d438db131bdf6e1ac6c78" member_type: VOTER last_known_addr { host: "127.21.178.190" port: 34091 } } peers { permanent_uuid: "6d7aeaee8b1c46d29a54da6cc92cbf3d" member_type: VOTER last_known_addr { host: "127.21.178.189" port: 33967 } } peers { permanent_uuid: "fe980ff120154749b98f3e31aba0cdd6" member_type: VOTER last_known_addr { host: "127.21.178.188" port: 36361 } } }
I20260812 08:03:50.772878 22475 sys_catalog.cc:458] T 00000000000000000000000000000000 P fe980ff120154749b98f3e31aba0cdd6 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 08:03:50.773425 22464 sys_catalog.cc:565] T 00000000000000000000000000000000 P fe980ff120154749b98f3e31aba0cdd6 [sys.catalog]: configured and running, proceeding with master startup.
W20260812 08:03:50.778399 22487 catalog_manager.cc:1594] T 00000000000000000000000000000000 P fe980ff120154749b98f3e31aba0cdd6: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 08:03:50.778506 22487 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 08:03:50.777099 22384 master_runner.cc:187] Error getting master registration for 127.21.178.190:34091: OK, instance_id { permanent_uuid: "43d0f953cc1d438db131bdf6e1ac6c78" instance_seqno: 1786521830156799 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260812 08:03:50.778909 22384 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260812 08:03:50.785688 22300 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.178.188" port: 36361 } attempt: 3
I20260812 08:03:50.792712 22300 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 43d0f953cc1d438db131bdf6e1ac6c78: Bootstrap starting.
I20260812 08:03:50.794994 22300 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 43d0f953cc1d438db131bdf6e1ac6c78: Neither blocks nor log segments found. Creating new log.
I20260812 08:03:50.795748 22300 log.cc:826] T 00000000000000000000000000000000 P 43d0f953cc1d438db131bdf6e1ac6c78: Log is configured to *not* fsync() on all Append() calls
I20260812 08:03:50.797631 22300 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 43d0f953cc1d438db131bdf6e1ac6c78: No bootstrap required, opened a new log
I20260812 08:03:50.801540 22300 raft_consensus.cc:359] T 00000000000000000000000000000000 P 43d0f953cc1d438db131bdf6e1ac6c78 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43d0f953cc1d438db131bdf6e1ac6c78" member_type: VOTER last_known_addr { host: "127.21.178.190" port: 34091 } } peers { permanent_uuid: "6d7aeaee8b1c46d29a54da6cc92cbf3d" member_type: VOTER last_known_addr { host: "127.21.178.189" port: 33967 } } peers { permanent_uuid: "fe980ff120154749b98f3e31aba0cdd6" member_type: VOTER last_known_addr { host: "127.21.178.188" port: 36361 } }
I20260812 08:03:50.801721 22300 raft_consensus.cc:385] T 00000000000000000000000000000000 P 43d0f953cc1d438db131bdf6e1ac6c78 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 08:03:50.801756 22300 raft_consensus.cc:740] T 00000000000000000000000000000000 P 43d0f953cc1d438db131bdf6e1ac6c78 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43d0f953cc1d438db131bdf6e1ac6c78, State: Initialized, Role: FOLLOWER
I20260812 08:03:50.802299 22300 consensus_queue.cc:260] T 00000000000000000000000000000000 P 43d0f953cc1d438db131bdf6e1ac6c78 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43d0f953cc1d438db131bdf6e1ac6c78" member_type: VOTER last_known_addr { host: "127.21.178.190" port: 34091 } } peers { permanent_uuid: "6d7aeaee8b1c46d29a54da6cc92cbf3d" member_type: VOTER last_known_addr { host: "127.21.178.189" port: 33967 } } peers { permanent_uuid: "fe980ff120154749b98f3e31aba0cdd6" member_type: VOTER last_known_addr { host: "127.21.178.188" port: 36361 } }
I20260812 08:03:50.802838 22490 sys_catalog.cc:455] T 00000000000000000000000000000000 P 43d0f953cc1d438db131bdf6e1ac6c78 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43d0f953cc1d438db131bdf6e1ac6c78" member_type: VOTER last_known_addr { host: "127.21.178.190" port: 34091 } } peers { permanent_uuid: "6d7aeaee8b1c46d29a54da6cc92cbf3d" member_type: VOTER last_known_addr { host: "127.21.178.189" port: 33967 } } peers { permanent_uuid: "fe980ff120154749b98f3e31aba0cdd6" member_type: VOTER last_known_addr { host: "127.21.178.188" port: 36361 } } }
I20260812 08:03:50.802971 22490 sys_catalog.cc:458] T 00000000000000000000000000000000 P 43d0f953cc1d438db131bdf6e1ac6c78 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 08:03:50.803637 22300 sys_catalog.cc:565] T 00000000000000000000000000000000 P 43d0f953cc1d438db131bdf6e1ac6c78 [sys.catalog]: configured and running, proceeding with master startup.
W20260812 08:03:50.811585 22504 catalog_manager.cc:1594] T 00000000000000000000000000000000 P 43d0f953cc1d438db131bdf6e1ac6c78: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 08:03:50.811686 22504 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 08:03:50.809827 22232 master_runner.cc:187] Error getting master registration for 127.21.178.189:33967: OK, instance_id { permanent_uuid: "6d7aeaee8b1c46d29a54da6cc92cbf3d" instance_seqno: 1786521830451435 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260812 08:03:50.812201 22232 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260812 08:03:50.838972 22380 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.21.178.188" port: 36361 } attempt: 4
I20260812 08:03:50.845459 22380 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6d7aeaee8b1c46d29a54da6cc92cbf3d: Bootstrap starting.
I20260812 08:03:50.847728 22380 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6d7aeaee8b1c46d29a54da6cc92cbf3d: Neither blocks nor log segments found. Creating new log.
I20260812 08:03:50.849941 22380 log.cc:826] T 00000000000000000000000000000000 P 6d7aeaee8b1c46d29a54da6cc92cbf3d: Log is configured to *not* fsync() on all Append() calls
I20260812 08:03:50.851974 22380 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6d7aeaee8b1c46d29a54da6cc92cbf3d: No bootstrap required, opened a new log
I20260812 08:03:50.854696 22380 raft_consensus.cc:359] T 00000000000000000000000000000000 P 6d7aeaee8b1c46d29a54da6cc92cbf3d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43d0f953cc1d438db131bdf6e1ac6c78" member_type: VOTER last_known_addr { host: "127.21.178.190" port: 34091 } } peers { permanent_uuid: "6d7aeaee8b1c46d29a54da6cc92cbf3d" member_type: VOTER last_known_addr { host: "127.21.178.189" port: 33967 } } peers { permanent_uuid: "fe980ff120154749b98f3e31aba0cdd6" member_type: VOTER last_known_addr { host: "127.21.178.188" port: 36361 } }
I20260812 08:03:50.854893 22380 raft_consensus.cc:385] T 00000000000000000000000000000000 P 6d7aeaee8b1c46d29a54da6cc92cbf3d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 08:03:50.854972 22380 raft_consensus.cc:740] T 00000000000000000000000000000000 P 6d7aeaee8b1c46d29a54da6cc92cbf3d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6d7aeaee8b1c46d29a54da6cc92cbf3d, State: Initialized, Role: FOLLOWER
I20260812 08:03:50.855479 22380 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6d7aeaee8b1c46d29a54da6cc92cbf3d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43d0f953cc1d438db131bdf6e1ac6c78" member_type: VOTER last_known_addr { host: "127.21.178.190" port: 34091 } } peers { permanent_uuid: "6d7aeaee8b1c46d29a54da6cc92cbf3d" member_type: VOTER last_known_addr { host: "127.21.178.189" port: 33967 } } peers { permanent_uuid: "fe980ff120154749b98f3e31aba0cdd6" member_type: VOTER last_known_addr { host: "127.21.178.188" port: 36361 } }
I20260812 08:03:50.856011 22507 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6d7aeaee8b1c46d29a54da6cc92cbf3d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43d0f953cc1d438db131bdf6e1ac6c78" member_type: VOTER last_known_addr { host: "127.21.178.190" port: 34091 } } peers { permanent_uuid: "6d7aeaee8b1c46d29a54da6cc92cbf3d" member_type: VOTER last_known_addr { host: "127.21.178.189" port: 33967 } } peers { permanent_uuid: "fe980ff120154749b98f3e31aba0cdd6" member_type: VOTER last_known_addr { host: "127.21.178.188" port: 36361 } } }
I20260812 08:03:50.856218 22507 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6d7aeaee8b1c46d29a54da6cc92cbf3d [sys.catalog]: This master's current role is: FOLLOWER
I20260812 08:03:50.856810 22380 sys_catalog.cc:565] T 00000000000000000000000000000000 P 6d7aeaee8b1c46d29a54da6cc92cbf3d [sys.catalog]: configured and running, proceeding with master startup.
W20260812 08:03:50.862188 22523 catalog_manager.cc:1594] T 00000000000000000000000000000000 P 6d7aeaee8b1c46d29a54da6cc92cbf3d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 08:03:50.862370 22523 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 08:03:50.868414 22305 master_runner.cc:264] No leader master found from master 6d7aeaee8b1c46d29a54da6cc92cbf3d
I20260812 08:03:50.868518 22305 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
I20260812 08:03:50.893453 22507 raft_consensus.cc:493] T 00000000000000000000000000000000 P 6d7aeaee8b1c46d29a54da6cc92cbf3d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 08:03:50.893713 22507 raft_consensus.cc:515] T 00000000000000000000000000000000 P 6d7aeaee8b1c46d29a54da6cc92cbf3d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43d0f953cc1d438db131bdf6e1ac6c78" member_type: VOTER last_known_addr { host: "127.21.178.190" port: 34091 } } peers { permanent_uuid: "6d7aeaee8b1c46d29a54da6cc92cbf3d" member_type: VOTER last_known_addr { host: "127.21.178.189" port: 33967 } } peers { permanent_uuid: "fe980ff120154749b98f3e31aba0cdd6" member_type: VOTER last_known_addr { host: "127.21.178.188" port: 36361 } }
I20260812 08:03:50.894948 22507 leader_election.cc:290] T 00000000000000000000000000000000 P 6d7aeaee8b1c46d29a54da6cc92cbf3d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 43d0f953cc1d438db131bdf6e1ac6c78 (127.21.178.190:34091), fe980ff120154749b98f3e31aba0cdd6 (127.21.178.188:36361)
I20260812 08:03:50.895955 22270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "6d7aeaee8b1c46d29a54da6cc92cbf3d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43d0f953cc1d438db131bdf6e1ac6c78" is_pre_election: true
I20260812 08:03:50.896488 22270 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 43d0f953cc1d438db131bdf6e1ac6c78 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6d7aeaee8b1c46d29a54da6cc92cbf3d in term 0.
I20260812 08:03:50.897213 22319 leader_election.cc:304] T 00000000000000000000000000000000 P 6d7aeaee8b1c46d29a54da6cc92cbf3d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 43d0f953cc1d438db131bdf6e1ac6c78, 6d7aeaee8b1c46d29a54da6cc92cbf3d; no voters: 
I20260812 08:03:50.898159 22431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "6d7aeaee8b1c46d29a54da6cc92cbf3d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fe980ff120154749b98f3e31aba0cdd6" is_pre_election: true
I20260812 08:03:50.898496 22431 raft_consensus.cc:2468] T 00000000000000000000000000000000 P fe980ff120154749b98f3e31aba0cdd6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6d7aeaee8b1c46d29a54da6cc92cbf3d in term 0.
I20260812 08:03:50.899266 22507 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 6d7aeaee8b1c46d29a54da6cc92cbf3d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 08:03:50.899367 22507 raft_consensus.cc:493] T 00000000000000000000000000000000 P 6d7aeaee8b1c46d29a54da6cc92cbf3d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 08:03:50.899417 22507 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 6d7aeaee8b1c46d29a54da6cc92cbf3d [term 0 FOLLOWER]: Advancing to term 1
I20260812 08:03:50.900789 22507 raft_consensus.cc:515] T 00000000000000000000000000000000 P 6d7aeaee8b1c46d29a54da6cc92cbf3d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43d0f953cc1d438db131bdf6e1ac6c78" member_type: VOTER last_known_addr { host: "127.21.178.190" port: 34091 } } peers { permanent_uuid: "6d7aeaee8b1c46d29a54da6cc92cbf3d" member_type: VOTER last_known_addr { host: "127.21.178.189" port: 33967 } } peers { permanent_uuid: "fe980ff120154749b98f3e31aba0cdd6" member_type: VOTER last_known_addr { host: "127.21.178.188" port: 36361 } }
I20260812 08:03:50.901225 22507 leader_election.cc:290] T 00000000000000000000000000000000 P 6d7aeaee8b1c46d29a54da6cc92cbf3d [CANDIDATE]: Term 1 election: Requested vote from peers 43d0f953cc1d438db131bdf6e1ac6c78 (127.21.178.190:34091), fe980ff120154749b98f3e31aba0cdd6 (127.21.178.188:36361)
I20260812 08:03:50.902124 22431 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "6d7aeaee8b1c46d29a54da6cc92cbf3d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fe980ff120154749b98f3e31aba0cdd6"
I20260812 08:03:50.902263 22431 raft_consensus.cc:3060] T 00000000000000000000000000000000 P fe980ff120154749b98f3e31aba0cdd6 [term 0 FOLLOWER]: Advancing to term 1
W20260812 08:03:50.902683 22248 tablet.cc:2406] T 00000000000000000000000000000000 P 43d0f953cc1d438db131bdf6e1ac6c78: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 08:03:50.903600 22431 raft_consensus.cc:2468] T 00000000000000000000000000000000 P fe980ff120154749b98f3e31aba0cdd6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6d7aeaee8b1c46d29a54da6cc92cbf3d in term 1.
I20260812 08:03:50.904083 22270 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "6d7aeaee8b1c46d29a54da6cc92cbf3d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43d0f953cc1d438db131bdf6e1ac6c78"
I20260812 08:03:50.904251 22270 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 43d0f953cc1d438db131bdf6e1ac6c78 [term 0 FOLLOWER]: Advancing to term 1
I20260812 08:03:50.905544 22270 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 43d0f953cc1d438db131bdf6e1ac6c78 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6d7aeaee8b1c46d29a54da6cc92cbf3d in term 1.
I20260812 08:03:50.906036 22319 leader_election.cc:304] T 00000000000000000000000000000000 P 6d7aeaee8b1c46d29a54da6cc92cbf3d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 43d0f953cc1d438db131bdf6e1ac6c78, 6d7aeaee8b1c46d29a54da6cc92cbf3d; no voters: 
I20260812 08:03:50.909287 22507 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 6d7aeaee8b1c46d29a54da6cc92cbf3d [term 1 FOLLOWER]: Leader election won for term 1
I20260812 08:03:50.909679 22507 raft_consensus.cc:697] T 00000000000000000000000000000000 P 6d7aeaee8b1c46d29a54da6cc92cbf3d [term 1 LEADER]: Becoming Leader. State: Replica: 6d7aeaee8b1c46d29a54da6cc92cbf3d, State: Running, Role: LEADER
I20260812 08:03:50.910116 22507 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6d7aeaee8b1c46d29a54da6cc92cbf3d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43d0f953cc1d438db131bdf6e1ac6c78" member_type: VOTER last_known_addr { host: "127.21.178.190" port: 34091 } } peers { permanent_uuid: "6d7aeaee8b1c46d29a54da6cc92cbf3d" member_type: VOTER last_known_addr { host: "127.21.178.189" port: 33967 } } peers { permanent_uuid: "fe980ff120154749b98f3e31aba0cdd6" member_type: VOTER last_known_addr { host: "127.21.178.188" port: 36361 } }
I20260812 08:03:50.912485 22507 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6d7aeaee8b1c46d29a54da6cc92cbf3d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6d7aeaee8b1c46d29a54da6cc92cbf3d. Latest consensus state: current_term: 1 leader_uuid: "6d7aeaee8b1c46d29a54da6cc92cbf3d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43d0f953cc1d438db131bdf6e1ac6c78" member_type: VOTER last_known_addr { host: "127.21.178.190" port: 34091 } } peers { permanent_uuid: "6d7aeaee8b1c46d29a54da6cc92cbf3d" member_type: VOTER last_known_addr { host: "127.21.178.189" port: 33967 } } peers { permanent_uuid: "fe980ff120154749b98f3e31aba0cdd6" member_type: VOTER last_known_addr { host: "127.21.178.188" port: 36361 } } }
I20260812 08:03:50.912679 22507 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6d7aeaee8b1c46d29a54da6cc92cbf3d [sys.catalog]: This master's current role is: LEADER
I20260812 08:03:50.915092 22529 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260812 08:03:50.916455 22529 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260812 08:03:50.922766 22431 raft_consensus.cc:1275] T 00000000000000000000000000000000 P fe980ff120154749b98f3e31aba0cdd6 [term 1 FOLLOWER]: Refusing update from remote peer 6d7aeaee8b1c46d29a54da6cc92cbf3d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 08:03:50.923794 22526 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 6d7aeaee8b1c46d29a54da6cc92cbf3d [LEADER]: Connected to new peer: Peer: permanent_uuid: "fe980ff120154749b98f3e31aba0cdd6" member_type: VOTER last_known_addr { host: "127.21.178.188" port: 36361 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 08:03:50.932411 22475 sys_catalog.cc:455] T 00000000000000000000000000000000 P fe980ff120154749b98f3e31aba0cdd6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6d7aeaee8b1c46d29a54da6cc92cbf3d. Latest consensus state: current_term: 1 leader_uuid: "6d7aeaee8b1c46d29a54da6cc92cbf3d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43d0f953cc1d438db131bdf6e1ac6c78" member_type: VOTER last_known_addr { host: "127.21.178.190" port: 34091 } } peers { permanent_uuid: "6d7aeaee8b1c46d29a54da6cc92cbf3d" member_type: VOTER last_known_addr { host: "127.21.178.189" port: 33967 } } peers { permanent_uuid: "fe980ff120154749b98f3e31aba0cdd6" member_type: VOTER last_known_addr { host: "127.21.178.188" port: 36361 } } }
I20260812 08:03:50.932546 22475 sys_catalog.cc:458] T 00000000000000000000000000000000 P fe980ff120154749b98f3e31aba0cdd6 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 08:03:50.934267 22270 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 43d0f953cc1d438db131bdf6e1ac6c78 [term 1 FOLLOWER]: Refusing update from remote peer 6d7aeaee8b1c46d29a54da6cc92cbf3d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 08:03:50.934994 22526 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 6d7aeaee8b1c46d29a54da6cc92cbf3d [LEADER]: Connected to new peer: Peer: permanent_uuid: "43d0f953cc1d438db131bdf6e1ac6c78" member_type: VOTER last_known_addr { host: "127.21.178.190" port: 34091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 08:03:50.939131 22490 sys_catalog.cc:455] T 00000000000000000000000000000000 P 43d0f953cc1d438db131bdf6e1ac6c78 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6d7aeaee8b1c46d29a54da6cc92cbf3d. Latest consensus state: current_term: 1 leader_uuid: "6d7aeaee8b1c46d29a54da6cc92cbf3d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43d0f953cc1d438db131bdf6e1ac6c78" member_type: VOTER last_known_addr { host: "127.21.178.190" port: 34091 } } peers { permanent_uuid: "6d7aeaee8b1c46d29a54da6cc92cbf3d" member_type: VOTER last_known_addr { host: "127.21.178.189" port: 33967 } } peers { permanent_uuid: "fe980ff120154749b98f3e31aba0cdd6" member_type: VOTER last_known_addr { host: "127.21.178.188" port: 36361 } } }
I20260812 08:03:50.939275 22490 sys_catalog.cc:458] T 00000000000000000000000000000000 P 43d0f953cc1d438db131bdf6e1ac6c78 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 08:03:50.943769 22526 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6d7aeaee8b1c46d29a54da6cc92cbf3d [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "6d7aeaee8b1c46d29a54da6cc92cbf3d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43d0f953cc1d438db131bdf6e1ac6c78" member_type: VOTER last_known_addr { host: "127.21.178.190" port: 34091 } } peers { permanent_uuid: "6d7aeaee8b1c46d29a54da6cc92cbf3d" member_type: VOTER last_known_addr { host: "127.21.178.189" port: 33967 } } peers { permanent_uuid: "fe980ff120154749b98f3e31aba0cdd6" member_type: VOTER last_known_addr { host: "127.21.178.188" port: 36361 } } }
I20260812 08:03:50.943923 22526 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6d7aeaee8b1c46d29a54da6cc92cbf3d [sys.catalog]: This master's current role is: LEADER
I20260812 08:03:50.945473 22490 sys_catalog.cc:455] T 00000000000000000000000000000000 P 43d0f953cc1d438db131bdf6e1ac6c78 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "6d7aeaee8b1c46d29a54da6cc92cbf3d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43d0f953cc1d438db131bdf6e1ac6c78" member_type: VOTER last_known_addr { host: "127.21.178.190" port: 34091 } } peers { permanent_uuid: "6d7aeaee8b1c46d29a54da6cc92cbf3d" member_type: VOTER last_known_addr { host: "127.21.178.189" port: 33967 } } peers { permanent_uuid: "fe980ff120154749b98f3e31aba0cdd6" member_type: VOTER last_known_addr { host: "127.21.178.188" port: 36361 } } }
I20260812 08:03:50.945602 22490 sys_catalog.cc:458] T 00000000000000000000000000000000 P 43d0f953cc1d438db131bdf6e1ac6c78 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 08:03:50.946144 22530 mvcc.cc:204] Tried to move back new op lower bound from 7317593419450572800 to 7317593419412754432. Current Snapshot: MvccSnapshot[applied={T|T < 7317593419450572800}]
I20260812 08:03:50.949432 22534 mvcc.cc:204] Tried to move back new op lower bound from 7317593419450572800 to 7317593419412754432. Current Snapshot: MvccSnapshot[applied={T|T < 7317593419450572800}]
I20260812 08:03:50.950531 22507 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6d7aeaee8b1c46d29a54da6cc92cbf3d [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "6d7aeaee8b1c46d29a54da6cc92cbf3d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43d0f953cc1d438db131bdf6e1ac6c78" member_type: VOTER last_known_addr { host: "127.21.178.190" port: 34091 } } peers { permanent_uuid: "6d7aeaee8b1c46d29a54da6cc92cbf3d" member_type: VOTER last_known_addr { host: "127.21.178.189" port: 33967 } } peers { permanent_uuid: "fe980ff120154749b98f3e31aba0cdd6" member_type: VOTER last_known_addr { host: "127.21.178.188" port: 36361 } } }
I20260812 08:03:50.950661 22507 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6d7aeaee8b1c46d29a54da6cc92cbf3d [sys.catalog]: This master's current role is: LEADER
I20260812 08:03:50.953301 22475 sys_catalog.cc:455] T 00000000000000000000000000000000 P fe980ff120154749b98f3e31aba0cdd6 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "6d7aeaee8b1c46d29a54da6cc92cbf3d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43d0f953cc1d438db131bdf6e1ac6c78" member_type: VOTER last_known_addr { host: "127.21.178.190" port: 34091 } } peers { permanent_uuid: "6d7aeaee8b1c46d29a54da6cc92cbf3d" member_type: VOTER last_known_addr { host: "127.21.178.189" port: 33967 } } peers { permanent_uuid: "fe980ff120154749b98f3e31aba0cdd6" member_type: VOTER last_known_addr { host: "127.21.178.188" port: 36361 } } }
I20260812 08:03:50.953450 22475 sys_catalog.cc:458] T 00000000000000000000000000000000 P fe980ff120154749b98f3e31aba0cdd6 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 08:03:50.958437 22532 mvcc.cc:204] Tried to move back new op lower bound from 7317593419450572800 to 7317593419412754432. Current Snapshot: MvccSnapshot[applied={T|T < 7317593419450572800}]
I20260812 08:03:50.959630 22529 catalog_manager.cc:1383] Generated new cluster ID: cf68c36522494160a7ebf067d47f34e3
I20260812 08:03:50.959704 22529 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260812 08:03:51.003849 22529 catalog_manager.cc:1406] Generated new certificate authority record
I20260812 08:03:51.005251 22529 catalog_manager.cc:1540] Loading token signing keys...
I20260812 08:03:51.023706 22529 catalog_manager.cc:6092] T 00000000000000000000000000000000 P 6d7aeaee8b1c46d29a54da6cc92cbf3d: Generated new TSK 0
I20260812 08:03:51.024601 22529 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260812 08:03:51.046377 22218 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskijWYiT/build/debug/bin/kudu
/tmp/dist-test-taskijWYiT/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskijWYiT/test-tmp/hybrid_clock-test.0.HybridClockTest.SlowClockInitialisation.1786521827826957-22218-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskijWYiT/test-tmp/hybrid_clock-test.0.HybridClockTest.SlowClockInitialisation.1786521827826957-22218-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskijWYiT/test-tmp/hybrid_clock-test.0.HybridClockTest.SlowClockInitialisation.1786521827826957-22218-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskijWYiT/test-tmp/hybrid_clock-test.0.HybridClockTest.SlowClockInitialisation.1786521827826957-22218-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.178.129:0
--local_ip_for_outbound_sockets=127.21.178.129
--webserver_interface=127.21.178.129
--webserver_port=0
--tserver_master_addrs=127.21.178.190:34091,127.21.178.189:33967,127.21.178.188:36361
--builtin_ntp_servers=127.21.178.148:45861
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--hybrid_clock_inject_init_delay_ms=100
--wall_clock_jump_detection=enabled with env {}
W20260812 08:03:51.170692 22539 flags.cc:432] Enabled unsafe flag: --hybrid_clock_inject_init_delay_ms=100
W20260812 08:03:51.170958 22539 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 08:03:51.170981 22539 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 08:03:51.171097 22539 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 08:03:51.174975 22539 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 08:03:51.175057 22539 flags.cc:432] Enabled experimental flag: --wall_clock_jump_detection=enabled
W20260812 08:03:51.175148 22539 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.178.129
I20260812 08:03:51.179289 22539 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskijWYiT/test-tmp/hybrid_clock-test.0.HybridClockTest.SlowClockInitialisation.1786521827826957-22218-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.178.148:45861
--hybrid_clock_inject_init_delay_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskijWYiT/test-tmp/hybrid_clock-test.0.HybridClockTest.SlowClockInitialisation.1786521827826957-22218-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskijWYiT/test-tmp/hybrid_clock-test.0.HybridClockTest.SlowClockInitialisation.1786521827826957-22218-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.178.129:0
--rpc_server_allow_ephemeral_ports=true
--wall_clock_jump_detection=enabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskijWYiT/test-tmp/hybrid_clock-test.0.HybridClockTest.SlowClockInitialisation.1786521827826957-22218-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.178.129
--webserver_port=0
--tserver_master_addrs=127.21.178.190:34091,127.21.178.189:33967,127.21.178.188:36361
--never_fsync=true
--heap_profile_path=/tmp/kudu.22539
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.178.129

Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 08:03:51.180572 22539 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 08:03:51.181749 22539 file_cache.cc:504] Constructed file cache file cache with capacity 419430
I20260812 08:03:51.186094 22539 server_base.cc:839] enabling wall clock jump detection
I20260812 08:03:51.186371 22539 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
W20260812 08:03:51.186540 22539 hybrid_clock.cc:606] Injecting 100ms delay in HybridClock initialization process
I20260812 08:03:51.287946 22539 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 08:03:51.289237 22539 hybrid_clock.cc:648] HybridClock initialized: now 1786521831289215 us; error 53 us; skew 500 ppm
I20260812 08:03:51.291913 22539 webserver.cc:533] Webserver started at http://127.21.178.129:46079/ using document root <none> and password file <none>
I20260812 08:03:51.292663 22539 fs_manager.cc:362] Metadata directory not provided
I20260812 08:03:51.292749 22539 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 08:03:51.292953 22539 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 08:03:51.294778 22539 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskijWYiT/test-tmp/hybrid_clock-test.0.HybridClockTest.SlowClockInitialisation.1786521827826957-22218-0/minicluster-data/ts-0/data/instance:
uuid: "6ecd0a55071d44968e2769ebe34f0036"
format_stamp: "Formatted at 2026-08-12 08:03:51 on dist-test-slave-8w9v"
I20260812 08:03:51.295341 22539 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskijWYiT/test-tmp/hybrid_clock-test.0.HybridClockTest.SlowClockInitialisation.1786521827826957-22218-0/minicluster-data/ts-0/wal/instance:
uuid: "6ecd0a55071d44968e2769ebe34f0036"
format_stamp: "Formatted at 2026-08-12 08:03:51 on dist-test-slave-8w9v"
I20260812 08:03:51.299494 22539 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.002s	sys 0.004s
I20260812 08:03:51.301911 22548 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:03:51.303038 22539 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.001s	sys 0.000s
I20260812 08:03:51.303170 22539 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskijWYiT/test-tmp/hybrid_clock-test.0.HybridClockTest.SlowClockInitialisation.1786521827826957-22218-0/minicluster-data/ts-0/data,/tmp/dist-test-taskijWYiT/test-tmp/hybrid_clock-test.0.HybridClockTest.SlowClockInitialisation.1786521827826957-22218-0/minicluster-data/ts-0/wal
uuid: "6ecd0a55071d44968e2769ebe34f0036"
format_stamp: "Formatted at 2026-08-12 08:03:51 on dist-test-slave-8w9v"
I20260812 08:03:51.303277 22539 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskijWYiT/test-tmp/hybrid_clock-test.0.HybridClockTest.SlowClockInitialisation.1786521827826957-22218-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskijWYiT/test-tmp/hybrid_clock-test.0.HybridClockTest.SlowClockInitialisation.1786521827826957-22218-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskijWYiT/test-tmp/hybrid_clock-test.0.HybridClockTest.SlowClockInitialisation.1786521827826957-22218-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 08:03:51.346163 22539 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:03:51.346930 22539 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 08:03:51.347091 22539 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:03:51.347646 22539 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260812 08:03:51.348769 22539 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 08:03:51.348824 22539 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:03:51.348872 22539 ts_tablet_manager.cc:616] Registered 0 tablets
I20260812 08:03:51.348888 22539 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:03:51.358798 22539 rpc_server.cc:307] RPC server started. Bound to: 127.21.178.129:44811
I20260812 08:03:51.358945 22680 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.21.178.129:44811 every 8 connection(s)
I20260812 08:03:51.359905 22539 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskijWYiT/test-tmp/hybrid_clock-test.0.HybridClockTest.SlowClockInitialisation.1786521827826957-22218-0/minicluster-data/ts-0/data/info.pb
I20260812 08:03:51.366855 22218 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskijWYiT/build/debug/bin/kudu as pid 22539
I20260812 08:03:51.366981 22218 external_mini_cluster.cc:1448] Reading /tmp/dist-test-taskijWYiT/test-tmp/hybrid_clock-test.0.HybridClockTest.SlowClockInitialisation.1786521827826957-22218-0/minicluster-data/ts-0/wal/instance
I20260812 08:03:51.370808 22685 heartbeater.cc:344] Connected to a master server at 127.21.178.189:33967
I20260812 08:03:51.370891 22683 heartbeater.cc:344] Connected to a master server at 127.21.178.190:34091
I20260812 08:03:51.371099 22685 heartbeater.cc:461] Registering TS with master...
I20260812 08:03:51.371098 22683 heartbeater.cc:461] Registering TS with master...
I20260812 08:03:51.371769 22685 heartbeater.cc:507] Master 127.21.178.189:33967 requested a full tablet report, sending...
I20260812 08:03:51.371771 22683 heartbeater.cc:507] Master 127.21.178.190:34091 requested a full tablet report, sending...
I20260812 08:03:51.372428 22682 heartbeater.cc:344] Connected to a master server at 127.21.178.188:36361
I20260812 08:03:51.372517 22682 heartbeater.cc:461] Registering TS with master...
I20260812 08:03:51.372723 22682 heartbeater.cc:507] Master 127.21.178.188:36361 requested a full tablet report, sending...
I20260812 08:03:51.373314 22334 ts_manager.cc:194] Registered new tserver with Master: 6ecd0a55071d44968e2769ebe34f0036 (127.21.178.129:44811)
I20260812 08:03:51.373414 22258 ts_manager.cc:194] Registered new tserver with Master: 6ecd0a55071d44968e2769ebe34f0036 (127.21.178.129:44811)
I20260812 08:03:51.373728 22419 ts_manager.cc:194] Registered new tserver with Master: 6ecd0a55071d44968e2769ebe34f0036 (127.21.178.129:44811)
I20260812 08:03:51.374717 22334 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.21.178.129:37651
I20260812 08:03:51.389603 22218 external_mini_cluster.cc:955] 1 TS(s) registered with all masters
I20260812 08:03:51.390172 22218 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskijWYiT/build/debug/bin/kudu with pid 22539
I20260812 08:03:51.401167 22218 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskijWYiT/build/debug/bin/kudu with pid 22232
I20260812 08:03:51.409710 22218 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskijWYiT/build/debug/bin/kudu with pid 22305
I20260812 08:03:51.419865 22218 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskijWYiT/build/debug/bin/kudu with pid 22384
I20260812 08:03:51.430955 22694 webui_checker.cc:102] curl thread will poll the following URLs every 0.001s:
  http://127.21.178.129:46079/transactions
  http://127.21.178.188:40415/metrics
  http://127.21.178.190:39985/metrics
  http://127.21.178.129:46079/metrics
  http://127.21.178.189:44687/metrics
I20260812 08:03:51.430965 22218 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskijWYiT/build/debug/bin/kudu
/tmp/dist-test-taskijWYiT/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskijWYiT/test-tmp/hybrid_clock-test.0.HybridClockTest.SlowClockInitialisation.1786521827826957-22218-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskijWYiT/test-tmp/hybrid_clock-test.0.HybridClockTest.SlowClockInitialisation.1786521827826957-22218-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskijWYiT/test-tmp/hybrid_clock-test.0.HybridClockTest.SlowClockInitialisation.1786521827826957-22218-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskijWYiT/test-tmp/hybrid_clock-test.0.HybridClockTest.SlowClockInitialisation.1786521827826957-22218-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.178.190:34091
--webserver_interface=127.21.178.190
--webserver_port=39985
--builtin_ntp_servers=127.21.178.148:45861
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.178.190:34091,127.21.178.189:33967,127.21.178.188:36361
--hybrid_clock_inject_init_delay_ms=100
--wall_clock_jump_detection=enabled with env {}
W20260812 08:03:51.539664 22695 flags.cc:432] Enabled unsafe flag: --hybrid_clock_inject_init_delay_ms=100
W20260812 08:03:51.539951 22695 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 08:03:51.539981 22695 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 08:03:51.540102 22695 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 08:03:51.543771 22695 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 08:03:51.543841 22695 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 08:03:51.543864 22695 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 08:03:51.543882 22695 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
W20260812 08:03:51.543907 22695 flags.cc:432] Enabled experimental flag: --wall_clock_jump_detection=enabled
I20260812 08:03:51.548241 22695 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskijWYiT/test-tmp/hybrid_clock-test.0.HybridClockTest.SlowClockInitialisation.1786521827826957-22218-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.178.148:45861
--hybrid_clock_inject_init_delay_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskijWYiT/test-tmp/hybrid_clock-test.0.HybridClockTest.SlowClockInitialisation.1786521827826957-22218-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskijWYiT/test-tmp/hybrid_clock-test.0.HybridClockTest.SlowClockInitialisation.1786521827826957-22218-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.178.190:34091,127.21.178.189:33967,127.21.178.188:36361
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.178.190:34091
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--wall_clock_jump_detection=enabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskijWYiT/test-tmp/hybrid_clock-test.0.HybridClockTest.SlowClockInitialisation.1786521827826957-22218-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.178.190
--webserver_port=39985
--never_fsync=true
--heap_profile_path=/tmp/kudu.22695
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false

Master server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 08:03:51.549427 22695 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 08:03:51.550645 22695 file_cache.cc:504] Constructed file cache file cache with capacity 419430
I20260812 08:03:51.553787 22695 server_base.cc:839] enabling wall clock jump detection
I20260812 08:03:51.554219 22695 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
W20260812 08:03:51.554412 22695 hybrid_clock.cc:606] Injecting 100ms delay in HybridClock initialization process
I20260812 08:03:51.656052 22695 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 08:03:52.168445 22695 hybrid_clock.cc:648] HybridClock initialized: now 1786521832168343 us; error 216 us; skew 500 ppm
I20260812 08:03:52.171296 22695 webserver.cc:533] Webserver started at http://127.21.178.190:39985/ using document root <none> and password file <none>
I20260812 08:03:52.172168 22695 fs_manager.cc:362] Metadata directory not provided
I20260812 08:03:52.172273 22695 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 08:03:52.176445 22695 fs_manager.cc:714] Time spent opening directory manager: real 0.003s	user 0.003s	sys 0.000s
I20260812 08:03:52.178857 22708 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:03:52.180189 22695 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
I20260812 08:03:52.180332 22695 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskijWYiT/test-tmp/hybrid_clock-test.0.HybridClockTest.SlowClockInitialisation.1786521827826957-22218-0/minicluster-data/master-0/data,/tmp/dist-test-taskijWYiT/test-tmp/hybrid_clock-test.0.HybridClockTest.SlowClockInitialisation.1786521827826957-22218-0/minicluster-data/master-0/wal
uuid: "43d0f953cc1d438db131bdf6e1ac6c78"
format_stamp: "Formatted at 2026-08-12 08:03:50 on dist-test-slave-8w9v"
I20260812 08:03:52.180835 22695 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskijWYiT/test-tmp/hybrid_clock-test.0.HybridClockTest.SlowClockInitialisation.1786521827826957-22218-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskijWYiT/test-tmp/hybrid_clock-test.0.HybridClockTest.SlowClockInitialisation.1786521827826957-22218-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskijWYiT/test-tmp/hybrid_clock-test.0.HybridClockTest.SlowClockInitialisation.1786521827826957-22218-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 08:03:52.196962 22695 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:03:52.197702 22695 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 08:03:52.197896 22695 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:03:52.207176 22778 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.21.178.190:34091 every 8 connection(s)
I20260812 08:03:52.207185 22695 rpc_server.cc:307] RPC server started. Bound to: 127.21.178.190:34091
I20260812 08:03:52.208757 22695 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskijWYiT/test-tmp/hybrid_clock-test.0.HybridClockTest.SlowClockInitialisation.1786521827826957-22218-0/minicluster-data/master-0/data/info.pb
I20260812 08:03:52.210920 22218 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskijWYiT/build/debug/bin/kudu as pid 22695
I20260812 08:03:52.211362 22218 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskijWYiT/build/debug/bin/kudu
/tmp/dist-test-taskijWYiT/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskijWYiT/test-tmp/hybrid_clock-test.0.HybridClockTest.SlowClockInitialisation.1786521827826957-22218-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskijWYiT/test-tmp/hybrid_clock-test.0.HybridClockTest.SlowClockInitialisation.1786521827826957-22218-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskijWYiT/test-tmp/hybrid_clock-test.0.HybridClockTest.SlowClockInitialisation.1786521827826957-22218-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskijWYiT/test-tmp/hybrid_clock-test.0.HybridClockTest.SlowClockInitialisation.1786521827826957-22218-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.178.189:33967
--webserver_interface=127.21.178.189
--webserver_port=44687
--builtin_ntp_servers=127.21.178.148:45861
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.178.190:34091,127.21.178.189:33967,127.21.178.188:36361
--hybrid_clock_inject_init_delay_ms=100
--wall_clock_jump_detection=enabled with env {}
I20260812 08:03:52.213984 22779 sys_catalog.cc:263] Verifying existing consensus state
I20260812 08:03:52.216073 22779 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 43d0f953cc1d438db131bdf6e1ac6c78: Bootstrap starting.
I20260812 08:03:52.234335 22779 log.cc:826] T 00000000000000000000000000000000 P 43d0f953cc1d438db131bdf6e1ac6c78: Log is configured to *not* fsync() on all Append() calls
I20260812 08:03:52.241886 22779 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 43d0f953cc1d438db131bdf6e1ac6c78: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 08:03:52.242389 22779 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 43d0f953cc1d438db131bdf6e1ac6c78: Bootstrap complete.
I20260812 08:03:52.247375 22779 raft_consensus.cc:359] T 00000000000000000000000000000000 P 43d0f953cc1d438db131bdf6e1ac6c78 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43d0f953cc1d438db131bdf6e1ac6c78" member_type: VOTER last_known_addr { host: "127.21.178.190" port: 34091 } } peers { permanent_uuid: "6d7aeaee8b1c46d29a54da6cc92cbf3d" member_type: VOTER last_known_addr { host: "127.21.178.189" port: 33967 } } peers { permanent_uuid: "fe980ff120154749b98f3e31aba0cdd6" member_type: VOTER last_known_addr { host: "127.21.178.188" port: 36361 } }
I20260812 08:03:52.247882 22779 raft_consensus.cc:740] T 00000000000000000000000000000000 P 43d0f953cc1d438db131bdf6e1ac6c78 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43d0f953cc1d438db131bdf6e1ac6c78, State: Initialized, Role: FOLLOWER
I20260812 08:03:52.248597 22779 consensus_queue.cc:260] T 00000000000000000000000000000000 P 43d0f953cc1d438db131bdf6e1ac6c78 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43d0f953cc1d438db131bdf6e1ac6c78" member_type: VOTER last_known_addr { host: "127.21.178.190" port: 34091 } } peers { permanent_uuid: "6d7aeaee8b1c46d29a54da6cc92cbf3d" member_type: VOTER last_known_addr { host: "127.21.178.189" port: 33967 } } peers { permanent_uuid: "fe980ff120154749b98f3e31aba0cdd6" member_type: VOTER last_known_addr { host: "127.21.178.188" port: 36361 } }
I20260812 08:03:52.249145 22784 sys_catalog.cc:455] T 00000000000000000000000000000000 P 43d0f953cc1d438db131bdf6e1ac6c78 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43d0f953cc1d438db131bdf6e1ac6c78" member_type: VOTER last_known_addr { host: "127.21.178.190" port: 34091 } } peers { permanent_uuid: "6d7aeaee8b1c46d29a54da6cc92cbf3d" member_type: VOTER last_known_addr { host: "127.21.178.189" port: 33967 } } peers { permanent_uuid: "fe980ff120154749b98f3e31aba0cdd6" member_type: VOTER last_known_addr { host: "127.21.178.188" port: 36361 } } }
I20260812 08:03:52.249347 22784 sys_catalog.cc:458] T 00000000000000000000000000000000 P 43d0f953cc1d438db131bdf6e1ac6c78 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 08:03:52.249910 22779 sys_catalog.cc:565] T 00000000000000000000000000000000 P 43d0f953cc1d438db131bdf6e1ac6c78 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 08:03:52.255229 22796 catalog_manager.cc:1295] Loaded cluster ID: cf68c36522494160a7ebf067d47f34e3
I20260812 08:03:52.255381 22796 catalog_manager.cc:1588] T 00000000000000000000000000000000 P 43d0f953cc1d438db131bdf6e1ac6c78: loading cluster ID for follower catalog manager: success
I20260812 08:03:52.258255 22796 catalog_manager.cc:1610] T 00000000000000000000000000000000 P 43d0f953cc1d438db131bdf6e1ac6c78: acquiring CA information for follower catalog manager: success
I20260812 08:03:52.259615 22796 catalog_manager.cc:1638] T 00000000000000000000000000000000 P 43d0f953cc1d438db131bdf6e1ac6c78: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260812 08:03:52.270700 22695 master_runner.cc:187] Error getting master registration for 127.21.178.189:33967: Network error: Client connection negotiation failed: client connection to 127.21.178.189:33967: connect: Connection refused (error 111), 
I20260812 08:03:52.274030 22695 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20260812 08:03:52.355211 22782 flags.cc:432] Enabled unsafe flag: --hybrid_clock_inject_init_delay_ms=100
W20260812 08:03:52.355566 22782 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 08:03:52.355659 22782 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 08:03:52.355762 22782 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 08:03:52.359427 22782 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 08:03:52.359546 22782 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 08:03:52.359586 22782 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 08:03:52.359680 22782 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
W20260812 08:03:52.359746 22782 flags.cc:432] Enabled experimental flag: --wall_clock_jump_detection=enabled
I20260812 08:03:52.364413 22782 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskijWYiT/test-tmp/hybrid_clock-test.0.HybridClockTest.SlowClockInitialisation.1786521827826957-22218-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.178.148:45861
--hybrid_clock_inject_init_delay_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskijWYiT/test-tmp/hybrid_clock-test.0.HybridClockTest.SlowClockInitialisation.1786521827826957-22218-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskijWYiT/test-tmp/hybrid_clock-test.0.HybridClockTest.SlowClockInitialisation.1786521827826957-22218-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.21.178.190:34091,127.21.178.189:33967,127.21.178.188:36361
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.178.189:33967
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--wall_clock_jump_detection=enabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskijWYiT/test-tmp/hybrid_clock-test.0.HybridClockTest.SlowClockInitialisation.1786521827826957-22218-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.21.178.189
--webserver_port=44687
--never_fsync=true
--heap_profile_path=/tmp/kudu.22782
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false

Master server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 08:03:52.365653 22782 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 08:03:52.366832 22782 file_cache.cc:504] Constructed file cache file cache with capacity 419430
I20260812 08:03:52.370177 22782 server_base.cc:839] enabling wall clock jump detection
I20260812 08:03:52.370558 22782 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
W20260812 08:03:52.370739 22782 hybrid_clock.cc:606] Injecting 100ms delay in HybridClock initialization process
I20260812 08:03:52.472597 22782 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 08:03:52.473838 22782 hybrid_clock.cc:648] HybridClock initialized: now 1786521832473809 us; error 50 us; skew 500 ppm
I20260812 08:03:52.476250 22782 webserver.cc:533] Webserver started at http://127.21.178.189:44687/ using document root <none> and password file <none>
I20260812 08:03:52.477090 22782 fs_manager.cc:362] Metadata directory not provided
I20260812 08:03:52.477201 22782 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 08:03:52.481761 22782 fs_manager.cc:714] Time spent opening directory manager: real 0.003s	user 0.004s	sys 0.000s
I20260812 08:03:52.484788 22808 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:03:52.486297 22782 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
I20260812 08:03:52.486466 22782 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskijWYiT/test-tmp/hybrid_clock-test.0.HybridClockTest.SlowClockInitialisation.1786521827826957-22218-0/minicluster-data/master-1/data,/tmp/dist-test-taskijWYiT/test-tmp/hybrid_clock-test.0.HybridClockTest.SlowClockInitialisation.1786521827826957-22218-0/minicluster-data/master-1/wal
uuid: "6d7aeaee8b1c46d29a54da6cc92cbf3d"
format_stamp: "Formatted at 2026-08-12 08:03:50 on dist-test-slave-8w9v"
I20260812 08:03:52.486994 22782 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskijWYiT/test-tmp/hybrid_clock-test.0.HybridClockTest.SlowClockInitialisation.1786521827826957-22218-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskijWYiT/test-tmp/hybrid_clock-test.0.HybridClockTest.SlowClockInitialisation.1786521827826957-22218-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskijWYiT/test-tmp/hybrid_clock-test.0.HybridClockTest.SlowClockInitialisation.1786521827826957-22218-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 08:03:52.519088 22782 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:03:52.519827 22782 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 08:03:52.520046 22782 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:03:52.529592 22782 rpc_server.cc:307] RPC server started. Bound to: 127.21.178.189:33967
I20260812 08:03:52.529606 22869 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.21.178.189:33967 every 8 connection(s)
I20260812 08:03:52.531029 22782 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskijWYiT/test-tmp/hybrid_clock-test.0.HybridClockTest.SlowClockInitialisation.1786521827826957-22218-0/minicluster-data/master-1/data/info.pb
I20260812 08:03:52.532815 22218 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskijWYiT/build/debug/bin/kudu as pid 22782
I20260812 08:03:52.533219 22218 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskijWYiT/build/debug/bin/kudu
/tmp/dist-test-taskijWYiT/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskijWYiT/test-tmp/hybrid_clock-test.0.HybridClockTest.SlowClockInitialisation.1786521827826957-22218-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskijWYiT/test-tmp/hybrid_clock-test.0.HybridClockTest.SlowClockInitialisation.1786521827826957-22218-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskijWYiT/test-tmp/hybrid_clock-test.0.HybridClockTest.SlowClockInitialisation.1786521827826957-22218-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskijWYiT/test-tmp/hybrid_clock-test.0.HybridClockTest.SlowClockInitialisation.1786521827826957-22218-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.178.188:36361
--webserver_interface=127.21.178.188
--webserver_port=40415
--builtin_ntp_servers=127.21.178.148:45861
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.178.190:34091,127.21.178.189:33967,127.21.178.188:36361
--hybrid_clock_inject_init_delay_ms=100
--wall_clock_jump_detection=enabled with env {}
I20260812 08:03:52.535962 22870 sys_catalog.cc:263] Verifying existing consensus state
I20260812 08:03:52.538813 22870 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6d7aeaee8b1c46d29a54da6cc92cbf3d: Bootstrap starting.
I20260812 08:03:52.558429 22870 log.cc:826] T 00000000000000000000000000000000 P 6d7aeaee8b1c46d29a54da6cc92cbf3d: Log is configured to *not* fsync() on all Append() calls
I20260812 08:03:52.566206 22870 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6d7aeaee8b1c46d29a54da6cc92cbf3d: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 08:03:52.566695 22870 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6d7aeaee8b1c46d29a54da6cc92cbf3d: Bootstrap complete.
I20260812 08:03:52.571506 22870 raft_consensus.cc:359] T 00000000000000000000000000000000 P 6d7aeaee8b1c46d29a54da6cc92cbf3d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43d0f953cc1d438db131bdf6e1ac6c78" member_type: VOTER last_known_addr { host: "127.21.178.190" port: 34091 } } peers { permanent_uuid: "6d7aeaee8b1c46d29a54da6cc92cbf3d" member_type: VOTER last_known_addr { host: "127.21.178.189" port: 33967 } } peers { permanent_uuid: "fe980ff120154749b98f3e31aba0cdd6" member_type: VOTER last_known_addr { host: "127.21.178.188" port: 36361 } }
I20260812 08:03:52.571954 22870 raft_consensus.cc:740] T 00000000000000000000000000000000 P 6d7aeaee8b1c46d29a54da6cc92cbf3d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6d7aeaee8b1c46d29a54da6cc92cbf3d, State: Initialized, Role: FOLLOWER
I20260812 08:03:52.572587 22870 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6d7aeaee8b1c46d29a54da6cc92cbf3d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43d0f953cc1d438db131bdf6e1ac6c78" member_type: VOTER last_known_addr { host: "127.21.178.190" port: 34091 } } peers { permanent_uuid: "6d7aeaee8b1c46d29a54da6cc92cbf3d" member_type: VOTER last_known_addr { host: "127.21.178.189" port: 33967 } } peers { permanent_uuid: "fe980ff120154749b98f3e31aba0cdd6" member_type: VOTER last_known_addr { host: "127.21.178.188" port: 36361 } }
I20260812 08:03:52.573136 22875 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6d7aeaee8b1c46d29a54da6cc92cbf3d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43d0f953cc1d438db131bdf6e1ac6c78" member_type: VOTER last_known_addr { host: "127.21.178.190" port: 34091 } } peers { permanent_uuid: "6d7aeaee8b1c46d29a54da6cc92cbf3d" member_type: VOTER last_known_addr { host: "127.21.178.189" port: 33967 } } peers { permanent_uuid: "fe980ff120154749b98f3e31aba0cdd6" member_type: VOTER last_known_addr { host: "127.21.178.188" port: 36361 } } }
I20260812 08:03:52.573333 22875 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6d7aeaee8b1c46d29a54da6cc92cbf3d [sys.catalog]: This master's current role is: FOLLOWER
I20260812 08:03:52.573901 22870 sys_catalog.cc:565] T 00000000000000000000000000000000 P 6d7aeaee8b1c46d29a54da6cc92cbf3d [sys.catalog]: configured and running, proceeding with master startup.
I20260812 08:03:52.579603 22888 catalog_manager.cc:1295] Loaded cluster ID: cf68c36522494160a7ebf067d47f34e3
I20260812 08:03:52.579751 22888 catalog_manager.cc:1588] T 00000000000000000000000000000000 P 6d7aeaee8b1c46d29a54da6cc92cbf3d: loading cluster ID for follower catalog manager: success
I20260812 08:03:52.582185 22888 catalog_manager.cc:1610] T 00000000000000000000000000000000 P 6d7aeaee8b1c46d29a54da6cc92cbf3d: acquiring CA information for follower catalog manager: success
I20260812 08:03:52.583530 22888 catalog_manager.cc:1638] T 00000000000000000000000000000000 P 6d7aeaee8b1c46d29a54da6cc92cbf3d: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260812 08:03:52.604671 22782 master_runner.cc:187] Error getting master registration for 127.21.178.188:36361: Network error: Client connection negotiation failed: client connection to 127.21.178.188:36361: connect: Connection refused (error 111), 
I20260812 08:03:52.606740 22782 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20260812 08:03:52.693912 22872 flags.cc:432] Enabled unsafe flag: --hybrid_clock_inject_init_delay_ms=100
W20260812 08:03:52.694286 22872 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 08:03:52.694339 22872 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 08:03:52.694415 22872 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 08:03:52.699302 22872 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 08:03:52.699407 22872 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 08:03:52.699440 22872 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 08:03:52.699467 22872 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
W20260812 08:03:52.699505 22872 flags.cc:432] Enabled experimental flag: --wall_clock_jump_detection=enabled
I20260812 08:03:52.705401 22872 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskijWYiT/test-tmp/hybrid_clock-test.0.HybridClockTest.SlowClockInitialisation.1786521827826957-22218-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.178.148:45861
--hybrid_clock_inject_init_delay_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskijWYiT/test-tmp/hybrid_clock-test.0.HybridClockTest.SlowClockInitialisation.1786521827826957-22218-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskijWYiT/test-tmp/hybrid_clock-test.0.HybridClockTest.SlowClockInitialisation.1786521827826957-22218-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.21.178.190:34091,127.21.178.189:33967,127.21.178.188:36361
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.178.188:36361
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--wall_clock_jump_detection=enabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskijWYiT/test-tmp/hybrid_clock-test.0.HybridClockTest.SlowClockInitialisation.1786521827826957-22218-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.21.178.188
--webserver_port=40415
--never_fsync=true
--heap_profile_path=/tmp/kudu.22872
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false

Master server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 08:03:52.706919 22872 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 08:03:52.708441 22872 file_cache.cc:504] Constructed file cache file cache with capacity 419430
I20260812 08:03:52.712392 22872 server_base.cc:839] enabling wall clock jump detection
I20260812 08:03:52.712817 22872 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
W20260812 08:03:52.712991 22872 hybrid_clock.cc:606] Injecting 100ms delay in HybridClock initialization process
I20260812 08:03:52.814679 22872 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 08:03:52.815892 22872 hybrid_clock.cc:648] HybridClock initialized: now 1786521832815875 us; error 51 us; skew 500 ppm
I20260812 08:03:52.818334 22872 webserver.cc:533] Webserver started at http://127.21.178.188:40415/ using document root <none> and password file <none>
I20260812 08:03:52.819154 22872 fs_manager.cc:362] Metadata directory not provided
I20260812 08:03:52.819275 22872 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 08:03:52.823308 22872 fs_manager.cc:714] Time spent opening directory manager: real 0.003s	user 0.002s	sys 0.000s
I20260812 08:03:52.825601 22907 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:03:52.826822 22872 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.001s	sys 0.000s
I20260812 08:03:52.826957 22872 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskijWYiT/test-tmp/hybrid_clock-test.0.HybridClockTest.SlowClockInitialisation.1786521827826957-22218-0/minicluster-data/master-2/data,/tmp/dist-test-taskijWYiT/test-tmp/hybrid_clock-test.0.HybridClockTest.SlowClockInitialisation.1786521827826957-22218-0/minicluster-data/master-2/wal
uuid: "fe980ff120154749b98f3e31aba0cdd6"
format_stamp: "Formatted at 2026-08-12 08:03:50 on dist-test-slave-8w9v"
I20260812 08:03:52.827378 22872 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskijWYiT/test-tmp/hybrid_clock-test.0.HybridClockTest.SlowClockInitialisation.1786521827826957-22218-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskijWYiT/test-tmp/hybrid_clock-test.0.HybridClockTest.SlowClockInitialisation.1786521827826957-22218-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskijWYiT/test-tmp/hybrid_clock-test.0.HybridClockTest.SlowClockInitialisation.1786521827826957-22218-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 08:03:52.869197 22872 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:03:52.869956 22872 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 08:03:52.870174 22872 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:03:52.879262 22872 rpc_server.cc:307] RPC server started. Bound to: 127.21.178.188:36361
I20260812 08:03:52.879287 22962 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.21.178.188:36361 every 8 connection(s)
I20260812 08:03:52.880561 22872 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskijWYiT/test-tmp/hybrid_clock-test.0.HybridClockTest.SlowClockInitialisation.1786521827826957-22218-0/minicluster-data/master-2/data/info.pb
I20260812 08:03:52.883595 22218 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskijWYiT/build/debug/bin/kudu as pid 22872
I20260812 08:03:52.884007 22218 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskijWYiT/build/debug/bin/kudu
/tmp/dist-test-taskijWYiT/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskijWYiT/test-tmp/hybrid_clock-test.0.HybridClockTest.SlowClockInitialisation.1786521827826957-22218-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskijWYiT/test-tmp/hybrid_clock-test.0.HybridClockTest.SlowClockInitialisation.1786521827826957-22218-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskijWYiT/test-tmp/hybrid_clock-test.0.HybridClockTest.SlowClockInitialisation.1786521827826957-22218-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskijWYiT/test-tmp/hybrid_clock-test.0.HybridClockTest.SlowClockInitialisation.1786521827826957-22218-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.178.129:44811
--local_ip_for_outbound_sockets=127.21.178.129
--tserver_master_addrs=127.21.178.190:34091,127.21.178.189:33967,127.21.178.188:36361
--webserver_port=46079
--webserver_interface=127.21.178.129
--builtin_ntp_servers=127.21.178.148:45861
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--hybrid_clock_inject_init_delay_ms=100
--wall_clock_jump_detection=enabled with env {}
I20260812 08:03:52.885839 22964 sys_catalog.cc:263] Verifying existing consensus state
I20260812 08:03:52.888209 22964 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fe980ff120154749b98f3e31aba0cdd6: Bootstrap starting.
I20260812 08:03:52.906422 22964 log.cc:826] T 00000000000000000000000000000000 P fe980ff120154749b98f3e31aba0cdd6: Log is configured to *not* fsync() on all Append() calls
I20260812 08:03:52.913456 22964 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fe980ff120154749b98f3e31aba0cdd6: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 08:03:52.913889 22964 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fe980ff120154749b98f3e31aba0cdd6: Bootstrap complete.
I20260812 08:03:52.918614 22964 raft_consensus.cc:359] T 00000000000000000000000000000000 P fe980ff120154749b98f3e31aba0cdd6 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43d0f953cc1d438db131bdf6e1ac6c78" member_type: VOTER last_known_addr { host: "127.21.178.190" port: 34091 } } peers { permanent_uuid: "6d7aeaee8b1c46d29a54da6cc92cbf3d" member_type: VOTER last_known_addr { host: "127.21.178.189" port: 33967 } } peers { permanent_uuid: "fe980ff120154749b98f3e31aba0cdd6" member_type: VOTER last_known_addr { host: "127.21.178.188" port: 36361 } }
I20260812 08:03:52.919049 22964 raft_consensus.cc:740] T 00000000000000000000000000000000 P fe980ff120154749b98f3e31aba0cdd6 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: fe980ff120154749b98f3e31aba0cdd6, State: Initialized, Role: FOLLOWER
I20260812 08:03:52.919631 22964 consensus_queue.cc:260] T 00000000000000000000000000000000 P fe980ff120154749b98f3e31aba0cdd6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43d0f953cc1d438db131bdf6e1ac6c78" member_type: VOTER last_known_addr { host: "127.21.178.190" port: 34091 } } peers { permanent_uuid: "6d7aeaee8b1c46d29a54da6cc92cbf3d" member_type: VOTER last_known_addr { host: "127.21.178.189" port: 33967 } } peers { permanent_uuid: "fe980ff120154749b98f3e31aba0cdd6" member_type: VOTER last_known_addr { host: "127.21.178.188" port: 36361 } }
I20260812 08:03:52.920096 22968 sys_catalog.cc:455] T 00000000000000000000000000000000 P fe980ff120154749b98f3e31aba0cdd6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43d0f953cc1d438db131bdf6e1ac6c78" member_type: VOTER last_known_addr { host: "127.21.178.190" port: 34091 } } peers { permanent_uuid: "6d7aeaee8b1c46d29a54da6cc92cbf3d" member_type: VOTER last_known_addr { host: "127.21.178.189" port: 33967 } } peers { permanent_uuid: "fe980ff120154749b98f3e31aba0cdd6" member_type: VOTER last_known_addr { host: "127.21.178.188" port: 36361 } } }
I20260812 08:03:52.920255 22968 sys_catalog.cc:458] T 00000000000000000000000000000000 P fe980ff120154749b98f3e31aba0cdd6 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 08:03:52.920822 22964 sys_catalog.cc:565] T 00000000000000000000000000000000 P fe980ff120154749b98f3e31aba0cdd6 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 08:03:52.925953 22980 catalog_manager.cc:1295] Loaded cluster ID: cf68c36522494160a7ebf067d47f34e3
I20260812 08:03:52.926030 22980 catalog_manager.cc:1588] T 00000000000000000000000000000000 P fe980ff120154749b98f3e31aba0cdd6: loading cluster ID for follower catalog manager: success
I20260812 08:03:52.928344 22980 catalog_manager.cc:1610] T 00000000000000000000000000000000 P fe980ff120154749b98f3e31aba0cdd6: acquiring CA information for follower catalog manager: success
I20260812 08:03:52.929648 22980 catalog_manager.cc:1638] T 00000000000000000000000000000000 P fe980ff120154749b98f3e31aba0cdd6: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260812 08:03:52.950783 22872 master_runner.cc:264] No leader master found from master fe980ff120154749b98f3e31aba0cdd6
I20260812 08:03:52.950902 22872 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20260812 08:03:53.019196 22966 flags.cc:432] Enabled unsafe flag: --hybrid_clock_inject_init_delay_ms=100
W20260812 08:03:53.019555 22966 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 08:03:53.019593 22966 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 08:03:53.019654 22966 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 08:03:53.024597 22966 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 08:03:53.024704 22966 flags.cc:432] Enabled experimental flag: --wall_clock_jump_detection=enabled
W20260812 08:03:53.024794 22966 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.178.129
I20260812 08:03:53.030572 22966 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskijWYiT/test-tmp/hybrid_clock-test.0.HybridClockTest.SlowClockInitialisation.1786521827826957-22218-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.178.148:45861
--hybrid_clock_inject_init_delay_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskijWYiT/test-tmp/hybrid_clock-test.0.HybridClockTest.SlowClockInitialisation.1786521827826957-22218-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskijWYiT/test-tmp/hybrid_clock-test.0.HybridClockTest.SlowClockInitialisation.1786521827826957-22218-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.178.129:44811
--rpc_server_allow_ephemeral_ports=true
--wall_clock_jump_detection=enabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskijWYiT/test-tmp/hybrid_clock-test.0.HybridClockTest.SlowClockInitialisation.1786521827826957-22218-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.178.129
--webserver_port=46079
--tserver_master_addrs=127.21.178.190:34091,127.21.178.189:33967,127.21.178.188:36361
--never_fsync=true
--heap_profile_path=/tmp/kudu.22966
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.178.129

Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 08:03:53.032022 22966 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 08:03:53.033525 22966 file_cache.cc:504] Constructed file cache file cache with capacity 419430
I20260812 08:03:53.038976 22966 server_base.cc:839] enabling wall clock jump detection
I20260812 08:03:53.039288 22966 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
W20260812 08:03:53.039472 22966 hybrid_clock.cc:606] Injecting 100ms delay in HybridClock initialization process
I20260812 08:03:53.140872 22966 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 08:03:53.142117 22966 hybrid_clock.cc:648] HybridClock initialized: now 1786521833142090 us; error 49 us; skew 500 ppm
I20260812 08:03:53.149065 22966 webserver.cc:533] Webserver started at http://127.21.178.129:46079/ using document root <none> and password file <none>
I20260812 08:03:53.149848 22966 fs_manager.cc:362] Metadata directory not provided
I20260812 08:03:53.149956 22966 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 08:03:53.154688 22966 fs_manager.cc:714] Time spent opening directory manager: real 0.003s	user 0.001s	sys 0.003s
I20260812 08:03:53.157155 22994 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:03:53.158541 22966 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.000s	sys 0.003s
I20260812 08:03:53.158672 22966 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskijWYiT/test-tmp/hybrid_clock-test.0.HybridClockTest.SlowClockInitialisation.1786521827826957-22218-0/minicluster-data/ts-0/data,/tmp/dist-test-taskijWYiT/test-tmp/hybrid_clock-test.0.HybridClockTest.SlowClockInitialisation.1786521827826957-22218-0/minicluster-data/ts-0/wal
uuid: "6ecd0a55071d44968e2769ebe34f0036"
format_stamp: "Formatted at 2026-08-12 08:03:51 on dist-test-slave-8w9v"
I20260812 08:03:53.159176 22966 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskijWYiT/test-tmp/hybrid_clock-test.0.HybridClockTest.SlowClockInitialisation.1786521827826957-22218-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskijWYiT/test-tmp/hybrid_clock-test.0.HybridClockTest.SlowClockInitialisation.1786521827826957-22218-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskijWYiT/test-tmp/hybrid_clock-test.0.HybridClockTest.SlowClockInitialisation.1786521827826957-22218-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 08:03:53.179324 22966 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:03:53.180199 22966 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 08:03:53.180393 22966 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:03:53.181090 22966 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260812 08:03:53.182308 22966 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 08:03:53.182368 22966 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:03:53.182421 22966 ts_tablet_manager.cc:616] Registered 0 tablets
I20260812 08:03:53.182451 22966 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:03:53.194381 22966 rpc_server.cc:307] RPC server started. Bound to: 127.21.178.129:44811
I20260812 08:03:53.194427 23128 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.21.178.129:44811 every 8 connection(s)
I20260812 08:03:53.195770 22966 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskijWYiT/test-tmp/hybrid_clock-test.0.HybridClockTest.SlowClockInitialisation.1786521827826957-22218-0/minicluster-data/ts-0/data/info.pb
I20260812 08:03:53.200501 22218 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskijWYiT/build/debug/bin/kudu as pid 22966
I20260812 08:03:53.208678 23131 heartbeater.cc:344] Connected to a master server at 127.21.178.190:34091
I20260812 08:03:53.209357 23131 heartbeater.cc:461] Registering TS with master...
I20260812 08:03:53.209424 23132 heartbeater.cc:344] Connected to a master server at 127.21.178.189:33967
I20260812 08:03:53.210269 23132 heartbeater.cc:461] Registering TS with master...
I20260812 08:03:53.211027 23129 heartbeater.cc:344] Connected to a master server at 127.21.178.188:36361
I20260812 08:03:53.211124 23129 heartbeater.cc:461] Registering TS with master...
I20260812 08:03:53.211198 23132 heartbeater.cc:507] Master 127.21.178.189:33967 requested a full tablet report, sending...
I20260812 08:03:53.211301 23129 heartbeater.cc:507] Master 127.21.178.188:36361 requested a full tablet report, sending...
I20260812 08:03:53.211863 23131 heartbeater.cc:507] Master 127.21.178.190:34091 requested a full tablet report, sending...
I20260812 08:03:53.213037 22926 ts_manager.cc:194] Registered new tserver with Master: 6ecd0a55071d44968e2769ebe34f0036 (127.21.178.129:44811)
I20260812 08:03:53.214160 22725 ts_manager.cc:194] Registered new tserver with Master: 6ecd0a55071d44968e2769ebe34f0036 (127.21.178.129:44811)
I20260812 08:03:53.216586 22828 ts_manager.cc:194] Registered new tserver with Master: 6ecd0a55071d44968e2769ebe34f0036 (127.21.178.129:44811)
I20260812 08:03:53.233974 22218 external_mini_cluster.cc:955] 1 TS(s) registered with all masters
I20260812 08:03:53.908272 23140 raft_consensus.cc:493] T 00000000000000000000000000000000 P 43d0f953cc1d438db131bdf6e1ac6c78 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 08:03:53.908527 23140 raft_consensus.cc:515] T 00000000000000000000000000000000 P 43d0f953cc1d438db131bdf6e1ac6c78 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43d0f953cc1d438db131bdf6e1ac6c78" member_type: VOTER last_known_addr { host: "127.21.178.190" port: 34091 } } peers { permanent_uuid: "6d7aeaee8b1c46d29a54da6cc92cbf3d" member_type: VOTER last_known_addr { host: "127.21.178.189" port: 33967 } } peers { permanent_uuid: "fe980ff120154749b98f3e31aba0cdd6" member_type: VOTER last_known_addr { host: "127.21.178.188" port: 36361 } }
I20260812 08:03:53.909926 23140 leader_election.cc:290] T 00000000000000000000000000000000 P 43d0f953cc1d438db131bdf6e1ac6c78 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6d7aeaee8b1c46d29a54da6cc92cbf3d (127.21.178.189:33967), fe980ff120154749b98f3e31aba0cdd6 (127.21.178.188:36361)
I20260812 08:03:53.914705 22937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "43d0f953cc1d438db131bdf6e1ac6c78" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "fe980ff120154749b98f3e31aba0cdd6" is_pre_election: true
I20260812 08:03:53.914829 22842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "43d0f953cc1d438db131bdf6e1ac6c78" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6d7aeaee8b1c46d29a54da6cc92cbf3d" is_pre_election: true
I20260812 08:03:53.915127 22937 raft_consensus.cc:2468] T 00000000000000000000000000000000 P fe980ff120154749b98f3e31aba0cdd6 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 43d0f953cc1d438db131bdf6e1ac6c78 in term 1.
I20260812 08:03:53.915125 22842 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 6d7aeaee8b1c46d29a54da6cc92cbf3d [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 43d0f953cc1d438db131bdf6e1ac6c78 in term 1.
I20260812 08:03:53.915650 22710 leader_election.cc:304] T 00000000000000000000000000000000 P 43d0f953cc1d438db131bdf6e1ac6c78 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 43d0f953cc1d438db131bdf6e1ac6c78, 6d7aeaee8b1c46d29a54da6cc92cbf3d; no voters: 
I20260812 08:03:53.915938 23140 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 43d0f953cc1d438db131bdf6e1ac6c78 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260812 08:03:53.916044 23140 raft_consensus.cc:493] T 00000000000000000000000000000000 P 43d0f953cc1d438db131bdf6e1ac6c78 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 08:03:53.916103 23140 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 43d0f953cc1d438db131bdf6e1ac6c78 [term 1 FOLLOWER]: Advancing to term 2
I20260812 08:03:53.918027 23140 raft_consensus.cc:515] T 00000000000000000000000000000000 P 43d0f953cc1d438db131bdf6e1ac6c78 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43d0f953cc1d438db131bdf6e1ac6c78" member_type: VOTER last_known_addr { host: "127.21.178.190" port: 34091 } } peers { permanent_uuid: "6d7aeaee8b1c46d29a54da6cc92cbf3d" member_type: VOTER last_known_addr { host: "127.21.178.189" port: 33967 } } peers { permanent_uuid: "fe980ff120154749b98f3e31aba0cdd6" member_type: VOTER last_known_addr { host: "127.21.178.188" port: 36361 } }
I20260812 08:03:53.918542 23140 leader_election.cc:290] T 00000000000000000000000000000000 P 43d0f953cc1d438db131bdf6e1ac6c78 [CANDIDATE]: Term 2 election: Requested vote from peers 6d7aeaee8b1c46d29a54da6cc92cbf3d (127.21.178.189:33967), fe980ff120154749b98f3e31aba0cdd6 (127.21.178.188:36361)
I20260812 08:03:53.918989 22842 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "43d0f953cc1d438db131bdf6e1ac6c78" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6d7aeaee8b1c46d29a54da6cc92cbf3d"
I20260812 08:03:53.919040 22937 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "43d0f953cc1d438db131bdf6e1ac6c78" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "fe980ff120154749b98f3e31aba0cdd6"
I20260812 08:03:53.919160 22937 raft_consensus.cc:3060] T 00000000000000000000000000000000 P fe980ff120154749b98f3e31aba0cdd6 [term 1 FOLLOWER]: Advancing to term 2
I20260812 08:03:53.919150 22842 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 6d7aeaee8b1c46d29a54da6cc92cbf3d [term 1 FOLLOWER]: Advancing to term 2
I20260812 08:03:53.921469 22937 raft_consensus.cc:2468] T 00000000000000000000000000000000 P fe980ff120154749b98f3e31aba0cdd6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 43d0f953cc1d438db131bdf6e1ac6c78 in term 2.
I20260812 08:03:53.921530 22842 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 6d7aeaee8b1c46d29a54da6cc92cbf3d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 43d0f953cc1d438db131bdf6e1ac6c78 in term 2.
I20260812 08:03:53.921855 22711 leader_election.cc:304] T 00000000000000000000000000000000 P 43d0f953cc1d438db131bdf6e1ac6c78 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 43d0f953cc1d438db131bdf6e1ac6c78, fe980ff120154749b98f3e31aba0cdd6; no voters: 
I20260812 08:03:53.922084 23140 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 43d0f953cc1d438db131bdf6e1ac6c78 [term 2 FOLLOWER]: Leader election won for term 2
I20260812 08:03:53.922359 23140 raft_consensus.cc:697] T 00000000000000000000000000000000 P 43d0f953cc1d438db131bdf6e1ac6c78 [term 2 LEADER]: Becoming Leader. State: Replica: 43d0f953cc1d438db131bdf6e1ac6c78, State: Running, Role: LEADER
I20260812 08:03:53.922709 23140 consensus_queue.cc:237] T 00000000000000000000000000000000 P 43d0f953cc1d438db131bdf6e1ac6c78 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43d0f953cc1d438db131bdf6e1ac6c78" member_type: VOTER last_known_addr { host: "127.21.178.190" port: 34091 } } peers { permanent_uuid: "6d7aeaee8b1c46d29a54da6cc92cbf3d" member_type: VOTER last_known_addr { host: "127.21.178.189" port: 33967 } } peers { permanent_uuid: "fe980ff120154749b98f3e31aba0cdd6" member_type: VOTER last_known_addr { host: "127.21.178.188" port: 36361 } }
I20260812 08:03:53.924908 23145 sys_catalog.cc:455] T 00000000000000000000000000000000 P 43d0f953cc1d438db131bdf6e1ac6c78 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 43d0f953cc1d438db131bdf6e1ac6c78. Latest consensus state: current_term: 2 leader_uuid: "43d0f953cc1d438db131bdf6e1ac6c78" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43d0f953cc1d438db131bdf6e1ac6c78" member_type: VOTER last_known_addr { host: "127.21.178.190" port: 34091 } } peers { permanent_uuid: "6d7aeaee8b1c46d29a54da6cc92cbf3d" member_type: VOTER last_known_addr { host: "127.21.178.189" port: 33967 } } peers { permanent_uuid: "fe980ff120154749b98f3e31aba0cdd6" member_type: VOTER last_known_addr { host: "127.21.178.188" port: 36361 } } }
I20260812 08:03:53.925048 23145 sys_catalog.cc:458] T 00000000000000000000000000000000 P 43d0f953cc1d438db131bdf6e1ac6c78 [sys.catalog]: This master's current role is: LEADER
I20260812 08:03:53.925396 23147 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260812 08:03:53.926522 23147 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260812 08:03:53.926955 23147 catalog_manager.cc:1295] Loaded cluster ID: cf68c36522494160a7ebf067d47f34e3
I20260812 08:03:53.927012 23147 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260812 08:03:53.927615 23147 catalog_manager.cc:1540] Loading token signing keys...
I20260812 08:03:53.928045 23147 catalog_manager.cc:6103] T 00000000000000000000000000000000 P 43d0f953cc1d438db131bdf6e1ac6c78: Loaded TSK: 0
I20260812 08:03:53.928783 23147 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260812 08:03:54.217042 22725 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.21.178.129:56925
I20260812 08:03:54.234455 22218 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskijWYiT/build/debug/bin/kudu with pid 22966
I20260812 08:03:54.244943 22218 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskijWYiT/build/debug/bin/kudu with pid 22695
I20260812 08:03:54.257529 22218 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskijWYiT/build/debug/bin/kudu with pid 22782
I20260812 08:03:54.268046 22218 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskijWYiT/build/debug/bin/kudu with pid 22872
I20260812 08:03:54.280527 22218 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskijWYiT/build/debug/bin/kudu
/tmp/dist-test-taskijWYiT/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskijWYiT/test-tmp/hybrid_clock-test.0.HybridClockTest.SlowClockInitialisation.1786521827826957-22218-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskijWYiT/test-tmp/hybrid_clock-test.0.HybridClockTest.SlowClockInitialisation.1786521827826957-22218-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskijWYiT/test-tmp/hybrid_clock-test.0.HybridClockTest.SlowClockInitialisation.1786521827826957-22218-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskijWYiT/test-tmp/hybrid_clock-test.0.HybridClockTest.SlowClockInitialisation.1786521827826957-22218-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.178.190:34091
--webserver_interface=127.21.178.190
--webserver_port=39985
--builtin_ntp_servers=127.21.178.148:45861
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.178.190:34091,127.21.178.189:33967,127.21.178.188:36361
--hybrid_clock_inject_init_delay_ms=100
--wall_clock_jump_detection=enabled with env {}
W20260812 08:03:54.426216 23148 flags.cc:432] Enabled unsafe flag: --hybrid_clock_inject_init_delay_ms=100
W20260812 08:03:54.426512 23148 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 08:03:54.426577 23148 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 08:03:54.426674 23148 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 08:03:54.430375 23148 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 08:03:54.430459 23148 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 08:03:54.430483 23148 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 08:03:54.430505 23148 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
W20260812 08:03:54.430581 23148 flags.cc:432] Enabled experimental flag: --wall_clock_jump_detection=enabled
I20260812 08:03:54.434924 23148 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskijWYiT/test-tmp/hybrid_clock-test.0.HybridClockTest.SlowClockInitialisation.1786521827826957-22218-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.178.148:45861
--hybrid_clock_inject_init_delay_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskijWYiT/test-tmp/hybrid_clock-test.0.HybridClockTest.SlowClockInitialisation.1786521827826957-22218-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskijWYiT/test-tmp/hybrid_clock-test.0.HybridClockTest.SlowClockInitialisation.1786521827826957-22218-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.21.178.190:34091,127.21.178.189:33967,127.21.178.188:36361
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.178.190:34091
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--wall_clock_jump_detection=enabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskijWYiT/test-tmp/hybrid_clock-test.0.HybridClockTest.SlowClockInitialisation.1786521827826957-22218-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.21.178.190
--webserver_port=39985
--never_fsync=true
--heap_profile_path=/tmp/kudu.23148
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false

Master server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 08:03:54.436291 23148 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 08:03:54.437532 23148 file_cache.cc:504] Constructed file cache file cache with capacity 419430
I20260812 08:03:54.440879 23148 server_base.cc:839] enabling wall clock jump detection
I20260812 08:03:54.441305 23148 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
W20260812 08:03:54.441504 23148 hybrid_clock.cc:606] Injecting 100ms delay in HybridClock initialization process
I20260812 08:03:54.543401 23148 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 08:03:54.544751 23148 hybrid_clock.cc:648] HybridClock initialized: now 1786521834544721 us; error 35 us; skew 500 ppm
I20260812 08:03:54.547508 23148 webserver.cc:533] Webserver started at http://127.21.178.190:39985/ using document root <none> and password file <none>
I20260812 08:03:54.548296 23148 fs_manager.cc:362] Metadata directory not provided
I20260812 08:03:54.548429 23148 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 08:03:54.553023 23148 fs_manager.cc:714] Time spent opening directory manager: real 0.003s	user 0.003s	sys 0.000s
I20260812 08:03:54.555335 23164 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:03:54.556664 23148 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.002s	sys 0.001s
I20260812 08:03:54.556804 23148 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskijWYiT/test-tmp/hybrid_clock-test.0.HybridClockTest.SlowClockInitialisation.1786521827826957-22218-0/minicluster-data/master-0/data,/tmp/dist-test-taskijWYiT/test-tmp/hybrid_clock-test.0.HybridClockTest.SlowClockInitialisation.1786521827826957-22218-0/minicluster-data/master-0/wal
uuid: "43d0f953cc1d438db131bdf6e1ac6c78"
format_stamp: "Formatted at 2026-08-12 08:03:50 on dist-test-slave-8w9v"
I20260812 08:03:54.557246 23148 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskijWYiT/test-tmp/hybrid_clock-test.0.HybridClockTest.SlowClockInitialisation.1786521827826957-22218-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskijWYiT/test-tmp/hybrid_clock-test.0.HybridClockTest.SlowClockInitialisation.1786521827826957-22218-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskijWYiT/test-tmp/hybrid_clock-test.0.HybridClockTest.SlowClockInitialisation.1786521827826957-22218-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 08:03:54.574928 23148 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:03:54.575728 23148 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 08:03:54.575953 23148 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:03:54.586114 23148 rpc_server.cc:307] RPC server started. Bound to: 127.21.178.190:34091
I20260812 08:03:54.586097 23226 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.21.178.190:34091 every 8 connection(s)
I20260812 08:03:54.587438 23148 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskijWYiT/test-tmp/hybrid_clock-test.0.HybridClockTest.SlowClockInitialisation.1786521827826957-22218-0/minicluster-data/master-0/data/info.pb
I20260812 08:03:54.590271 22218 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskijWYiT/build/debug/bin/kudu as pid 23148
I20260812 08:03:54.590731 22218 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskijWYiT/build/debug/bin/kudu
/tmp/dist-test-taskijWYiT/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskijWYiT/test-tmp/hybrid_clock-test.0.HybridClockTest.SlowClockInitialisation.1786521827826957-22218-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskijWYiT/test-tmp/hybrid_clock-test.0.HybridClockTest.SlowClockInitialisation.1786521827826957-22218-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskijWYiT/test-tmp/hybrid_clock-test.0.HybridClockTest.SlowClockInitialisation.1786521827826957-22218-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskijWYiT/test-tmp/hybrid_clock-test.0.HybridClockTest.SlowClockInitialisation.1786521827826957-22218-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.178.189:33967
--webserver_interface=127.21.178.189
--webserver_port=44687
--builtin_ntp_servers=127.21.178.148:45861
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.178.190:34091,127.21.178.189:33967,127.21.178.188:36361
--hybrid_clock_inject_init_delay_ms=100
--wall_clock_jump_detection=enabled with env {}
I20260812 08:03:54.591881 23227 sys_catalog.cc:263] Verifying existing consensus state
I20260812 08:03:54.594230 23227 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 43d0f953cc1d438db131bdf6e1ac6c78: Bootstrap starting.
I20260812 08:03:54.614750 23227 log.cc:826] T 00000000000000000000000000000000 P 43d0f953cc1d438db131bdf6e1ac6c78: Log is configured to *not* fsync() on all Append() calls
I20260812 08:03:54.622468 23227 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 43d0f953cc1d438db131bdf6e1ac6c78: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 1 replicates
I20260812 08:03:54.622982 23227 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 43d0f953cc1d438db131bdf6e1ac6c78: Bootstrap complete.
I20260812 08:03:54.627894 23227 raft_consensus.cc:359] T 00000000000000000000000000000000 P 43d0f953cc1d438db131bdf6e1ac6c78 [term 2 FOLLOWER]: Replica starting. Triggering 1 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43d0f953cc1d438db131bdf6e1ac6c78" member_type: VOTER last_known_addr { host: "127.21.178.190" port: 34091 } } peers { permanent_uuid: "6d7aeaee8b1c46d29a54da6cc92cbf3d" member_type: VOTER last_known_addr { host: "127.21.178.189" port: 33967 } } peers { permanent_uuid: "fe980ff120154749b98f3e31aba0cdd6" member_type: VOTER last_known_addr { host: "127.21.178.188" port: 36361 } }
I20260812 08:03:54.628599 23227 raft_consensus.cc:740] T 00000000000000000000000000000000 P 43d0f953cc1d438db131bdf6e1ac6c78 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43d0f953cc1d438db131bdf6e1ac6c78, State: Initialized, Role: FOLLOWER
I20260812 08:03:54.629215 23227 consensus_queue.cc:260] T 00000000000000000000000000000000 P 43d0f953cc1d438db131bdf6e1ac6c78 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 2.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43d0f953cc1d438db131bdf6e1ac6c78" member_type: VOTER last_known_addr { host: "127.21.178.190" port: 34091 } } peers { permanent_uuid: "6d7aeaee8b1c46d29a54da6cc92cbf3d" member_type: VOTER last_known_addr { host: "127.21.178.189" port: 33967 } } peers { permanent_uuid: "fe980ff120154749b98f3e31aba0cdd6" member_type: VOTER last_known_addr { host: "127.21.178.188" port: 36361 } }
I20260812 08:03:54.629757 23231 sys_catalog.cc:455] T 00000000000000000000000000000000 P 43d0f953cc1d438db131bdf6e1ac6c78 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43d0f953cc1d438db131bdf6e1ac6c78" member_type: VOTER last_known_addr { host: "127.21.178.190" port: 34091 } } peers { permanent_uuid: "6d7aeaee8b1c46d29a54da6cc92cbf3d" member_type: VOTER last_known_addr { host: "127.21.178.189" port: 33967 } } peers { permanent_uuid: "fe980ff120154749b98f3e31aba0cdd6" member_type: VOTER last_known_addr { host: "127.21.178.188" port: 36361 } } }
I20260812 08:03:54.629962 23231 sys_catalog.cc:458] T 00000000000000000000000000000000 P 43d0f953cc1d438db131bdf6e1ac6c78 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 08:03:54.630482 23227 sys_catalog.cc:565] T 00000000000000000000000000000000 P 43d0f953cc1d438db131bdf6e1ac6c78 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 08:03:54.635861 23243 catalog_manager.cc:1295] Loaded cluster ID: cf68c36522494160a7ebf067d47f34e3
I20260812 08:03:54.636015 23243 catalog_manager.cc:1588] T 00000000000000000000000000000000 P 43d0f953cc1d438db131bdf6e1ac6c78: loading cluster ID for follower catalog manager: success
I20260812 08:03:54.638453 23243 catalog_manager.cc:1610] T 00000000000000000000000000000000 P 43d0f953cc1d438db131bdf6e1ac6c78: acquiring CA information for follower catalog manager: success
I20260812 08:03:54.639793 23243 catalog_manager.cc:1638] T 00000000000000000000000000000000 P 43d0f953cc1d438db131bdf6e1ac6c78: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260812 08:03:54.646307 23148 master_runner.cc:187] Error getting master registration for 127.21.178.189:33967: Network error: Client connection negotiation failed: client connection to 127.21.178.189:33967: connect: Connection refused (error 111), 
I20260812 08:03:54.648495 23148 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20260812 08:03:54.746017 23229 flags.cc:432] Enabled unsafe flag: --hybrid_clock_inject_init_delay_ms=100
W20260812 08:03:54.746393 23229 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 08:03:54.746501 23229 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 08:03:54.746579 23229 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 08:03:54.751461 23229 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 08:03:54.751561 23229 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 08:03:54.751597 23229 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 08:03:54.751634 23229 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
W20260812 08:03:54.751745 23229 flags.cc:432] Enabled experimental flag: --wall_clock_jump_detection=enabled
I20260812 08:03:54.757689 23229 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskijWYiT/test-tmp/hybrid_clock-test.0.HybridClockTest.SlowClockInitialisation.1786521827826957-22218-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.178.148:45861
--hybrid_clock_inject_init_delay_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskijWYiT/test-tmp/hybrid_clock-test.0.HybridClockTest.SlowClockInitialisation.1786521827826957-22218-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskijWYiT/test-tmp/hybrid_clock-test.0.HybridClockTest.SlowClockInitialisation.1786521827826957-22218-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.21.178.190:34091,127.21.178.189:33967,127.21.178.188:36361
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.178.189:33967
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--wall_clock_jump_detection=enabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskijWYiT/test-tmp/hybrid_clock-test.0.HybridClockTest.SlowClockInitialisation.1786521827826957-22218-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.21.178.189
--webserver_port=44687
--never_fsync=true
--heap_profile_path=/tmp/kudu.23229
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false

Master server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 08:03:54.759214 23229 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 08:03:54.760766 23229 file_cache.cc:504] Constructed file cache file cache with capacity 419430
I20260812 08:03:54.764950 23229 server_base.cc:839] enabling wall clock jump detection
I20260812 08:03:54.765422 23229 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
W20260812 08:03:54.765614 23229 hybrid_clock.cc:606] Injecting 100ms delay in HybridClock initialization process
I20260812 08:03:54.867333 23229 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 08:03:54.868557 23229 hybrid_clock.cc:648] HybridClock initialized: now 1786521834868534 us; error 46 us; skew 500 ppm
I20260812 08:03:54.871026 23229 webserver.cc:533] Webserver started at http://127.21.178.189:44687/ using document root <none> and password file <none>
I20260812 08:03:54.871807 23229 fs_manager.cc:362] Metadata directory not provided
I20260812 08:03:54.871876 23229 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 08:03:54.876158 23229 fs_manager.cc:714] Time spent opening directory manager: real 0.003s	user 0.003s	sys 0.000s
I20260812 08:03:54.878648 23258 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:03:54.879935 23229 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260812 08:03:54.880067 23229 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskijWYiT/test-tmp/hybrid_clock-test.0.HybridClockTest.SlowClockInitialisation.1786521827826957-22218-0/minicluster-data/master-1/data,/tmp/dist-test-taskijWYiT/test-tmp/hybrid_clock-test.0.HybridClockTest.SlowClockInitialisation.1786521827826957-22218-0/minicluster-data/master-1/wal
uuid: "6d7aeaee8b1c46d29a54da6cc92cbf3d"
format_stamp: "Formatted at 2026-08-12 08:03:50 on dist-test-slave-8w9v"
I20260812 08:03:54.880609 23229 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskijWYiT/test-tmp/hybrid_clock-test.0.HybridClockTest.SlowClockInitialisation.1786521827826957-22218-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskijWYiT/test-tmp/hybrid_clock-test.0.HybridClockTest.SlowClockInitialisation.1786521827826957-22218-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskijWYiT/test-tmp/hybrid_clock-test.0.HybridClockTest.SlowClockInitialisation.1786521827826957-22218-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 08:03:54.907441 23229 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:03:54.908424 23229 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 08:03:54.908645 23229 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:03:54.918634 23316 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.21.178.189:33967 every 8 connection(s)
I20260812 08:03:54.918635 23229 rpc_server.cc:307] RPC server started. Bound to: 127.21.178.189:33967
I20260812 08:03:54.919934 23229 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskijWYiT/test-tmp/hybrid_clock-test.0.HybridClockTest.SlowClockInitialisation.1786521827826957-22218-0/minicluster-data/master-1/data/info.pb
I20260812 08:03:54.920650 22218 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskijWYiT/build/debug/bin/kudu as pid 23229
I20260812 08:03:54.921054 22218 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskijWYiT/build/debug/bin/kudu
/tmp/dist-test-taskijWYiT/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskijWYiT/test-tmp/hybrid_clock-test.0.HybridClockTest.SlowClockInitialisation.1786521827826957-22218-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskijWYiT/test-tmp/hybrid_clock-test.0.HybridClockTest.SlowClockInitialisation.1786521827826957-22218-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskijWYiT/test-tmp/hybrid_clock-test.0.HybridClockTest.SlowClockInitialisation.1786521827826957-22218-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskijWYiT/test-tmp/hybrid_clock-test.0.HybridClockTest.SlowClockInitialisation.1786521827826957-22218-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.178.188:36361
--webserver_interface=127.21.178.188
--webserver_port=40415
--builtin_ntp_servers=127.21.178.148:45861
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.21.178.190:34091,127.21.178.189:33967,127.21.178.188:36361
--hybrid_clock_inject_init_delay_ms=100
--wall_clock_jump_detection=enabled with env {}
I20260812 08:03:54.924858 23318 sys_catalog.cc:263] Verifying existing consensus state
I20260812 08:03:54.927073 23318 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6d7aeaee8b1c46d29a54da6cc92cbf3d: Bootstrap starting.
I20260812 08:03:54.945237 23318 log.cc:826] T 00000000000000000000000000000000 P 6d7aeaee8b1c46d29a54da6cc92cbf3d: Log is configured to *not* fsync() on all Append() calls
I20260812 08:03:54.952997 23318 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6d7aeaee8b1c46d29a54da6cc92cbf3d: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 08:03:54.953483 23318 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6d7aeaee8b1c46d29a54da6cc92cbf3d: Bootstrap complete.
I20260812 08:03:54.958278 23318 raft_consensus.cc:359] T 00000000000000000000000000000000 P 6d7aeaee8b1c46d29a54da6cc92cbf3d [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43d0f953cc1d438db131bdf6e1ac6c78" member_type: VOTER last_known_addr { host: "127.21.178.190" port: 34091 } } peers { permanent_uuid: "6d7aeaee8b1c46d29a54da6cc92cbf3d" member_type: VOTER last_known_addr { host: "127.21.178.189" port: 33967 } } peers { permanent_uuid: "fe980ff120154749b98f3e31aba0cdd6" member_type: VOTER last_known_addr { host: "127.21.178.188" port: 36361 } }
I20260812 08:03:54.958710 23318 raft_consensus.cc:740] T 00000000000000000000000000000000 P 6d7aeaee8b1c46d29a54da6cc92cbf3d [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6d7aeaee8b1c46d29a54da6cc92cbf3d, State: Initialized, Role: FOLLOWER
I20260812 08:03:54.959324 23318 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6d7aeaee8b1c46d29a54da6cc92cbf3d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43d0f953cc1d438db131bdf6e1ac6c78" member_type: VOTER last_known_addr { host: "127.21.178.190" port: 34091 } } peers { permanent_uuid: "6d7aeaee8b1c46d29a54da6cc92cbf3d" member_type: VOTER last_known_addr { host: "127.21.178.189" port: 33967 } } peers { permanent_uuid: "fe980ff120154749b98f3e31aba0cdd6" member_type: VOTER last_known_addr { host: "127.21.178.188" port: 36361 } }
I20260812 08:03:54.960386 23323 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6d7aeaee8b1c46d29a54da6cc92cbf3d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43d0f953cc1d438db131bdf6e1ac6c78" member_type: VOTER last_known_addr { host: "127.21.178.190" port: 34091 } } peers { permanent_uuid: "6d7aeaee8b1c46d29a54da6cc92cbf3d" member_type: VOTER last_known_addr { host: "127.21.178.189" port: 33967 } } peers { permanent_uuid: "fe980ff120154749b98f3e31aba0cdd6" member_type: VOTER last_known_addr { host: "127.21.178.188" port: 36361 } } }
I20260812 08:03:54.960541 23323 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6d7aeaee8b1c46d29a54da6cc92cbf3d [sys.catalog]: This master's current role is: FOLLOWER
I20260812 08:03:54.960660 23318 sys_catalog.cc:565] T 00000000000000000000000000000000 P 6d7aeaee8b1c46d29a54da6cc92cbf3d [sys.catalog]: configured and running, proceeding with master startup.
I20260812 08:03:54.966917 23335 catalog_manager.cc:1295] Loaded cluster ID: cf68c36522494160a7ebf067d47f34e3
I20260812 08:03:54.967077 23335 catalog_manager.cc:1588] T 00000000000000000000000000000000 P 6d7aeaee8b1c46d29a54da6cc92cbf3d: loading cluster ID for follower catalog manager: success
I20260812 08:03:54.969400 23335 catalog_manager.cc:1610] T 00000000000000000000000000000000 P 6d7aeaee8b1c46d29a54da6cc92cbf3d: acquiring CA information for follower catalog manager: success
I20260812 08:03:54.970734 23335 catalog_manager.cc:1638] T 00000000000000000000000000000000 P 6d7aeaee8b1c46d29a54da6cc92cbf3d: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260812 08:03:54.983932 23229 master_runner.cc:187] Error getting master registration for 127.21.178.188:36361: Network error: Client connection negotiation failed: client connection to 127.21.178.188:36361: connect: Connection refused (error 111), 
I20260812 08:03:54.986004 23229 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20260812 08:03:55.049901 23321 flags.cc:432] Enabled unsafe flag: --hybrid_clock_inject_init_delay_ms=100
W20260812 08:03:55.050271 23321 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 08:03:55.050361 23321 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 08:03:55.050457 23321 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 08:03:55.054298 23321 flags.cc:432] Enabled experimental flag: --ipki_ca_key_size=768
W20260812 08:03:55.054405 23321 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 08:03:55.054456 23321 flags.cc:432] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260812 08:03:55.054495 23321 flags.cc:432] Enabled experimental flag: --rpc_reuseport=true
W20260812 08:03:55.054540 23321 flags.cc:432] Enabled experimental flag: --wall_clock_jump_detection=enabled
I20260812 08:03:55.058887 23321 master_runner.cc:387] Master server non-default flags:
--log_dir=/tmp/dist-test-taskijWYiT/test-tmp/hybrid_clock-test.0.HybridClockTest.SlowClockInitialisation.1786521827826957-22218-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.178.148:45861
--hybrid_clock_inject_init_delay_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskijWYiT/test-tmp/hybrid_clock-test.0.HybridClockTest.SlowClockInitialisation.1786521827826957-22218-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskijWYiT/test-tmp/hybrid_clock-test.0.HybridClockTest.SlowClockInitialisation.1786521827826957-22218-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.21.178.190:34091,127.21.178.189:33967,127.21.178.188:36361
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.21.178.188:36361
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--wall_clock_jump_detection=enabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskijWYiT/test-tmp/hybrid_clock-test.0.HybridClockTest.SlowClockInitialisation.1786521827826957-22218-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.21.178.188
--webserver_port=40415
--never_fsync=true
--heap_profile_path=/tmp/kudu.23321
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false

Master server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 08:03:55.060164 23321 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 08:03:55.061492 23321 file_cache.cc:504] Constructed file cache file cache with capacity 419430
I20260812 08:03:55.065088 23321 server_base.cc:839] enabling wall clock jump detection
I20260812 08:03:55.065491 23321 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
W20260812 08:03:55.065675 23321 hybrid_clock.cc:606] Injecting 100ms delay in HybridClock initialization process
I20260812 08:03:55.167508 23321 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 08:03:55.168710 23321 hybrid_clock.cc:648] HybridClock initialized: now 1786521835168682 us; error 64 us; skew 500 ppm
I20260812 08:03:55.171417 23321 webserver.cc:533] Webserver started at http://127.21.178.188:40415/ using document root <none> and password file <none>
I20260812 08:03:55.172175 23321 fs_manager.cc:362] Metadata directory not provided
I20260812 08:03:55.172252 23321 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 08:03:55.176285 23321 fs_manager.cc:714] Time spent opening directory manager: real 0.003s	user 0.004s	sys 0.000s
I20260812 08:03:55.178543 23350 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:03:55.179829 23321 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260812 08:03:55.179988 23321 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskijWYiT/test-tmp/hybrid_clock-test.0.HybridClockTest.SlowClockInitialisation.1786521827826957-22218-0/minicluster-data/master-2/data,/tmp/dist-test-taskijWYiT/test-tmp/hybrid_clock-test.0.HybridClockTest.SlowClockInitialisation.1786521827826957-22218-0/minicluster-data/master-2/wal
uuid: "fe980ff120154749b98f3e31aba0cdd6"
format_stamp: "Formatted at 2026-08-12 08:03:50 on dist-test-slave-8w9v"
I20260812 08:03:55.180507 23321 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskijWYiT/test-tmp/hybrid_clock-test.0.HybridClockTest.SlowClockInitialisation.1786521827826957-22218-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskijWYiT/test-tmp/hybrid_clock-test.0.HybridClockTest.SlowClockInitialisation.1786521827826957-22218-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskijWYiT/test-tmp/hybrid_clock-test.0.HybridClockTest.SlowClockInitialisation.1786521827826957-22218-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 08:03:55.207331 23321 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:03:55.208271 23321 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 08:03:55.208503 23321 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:03:55.217763 23410 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.21.178.188:36361 every 8 connection(s)
I20260812 08:03:55.217769 23321 rpc_server.cc:307] RPC server started. Bound to: 127.21.178.188:36361
I20260812 08:03:55.219323 23321 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskijWYiT/test-tmp/hybrid_clock-test.0.HybridClockTest.SlowClockInitialisation.1786521827826957-22218-0/minicluster-data/master-2/data/info.pb
I20260812 08:03:55.220736 22218 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskijWYiT/build/debug/bin/kudu as pid 23321
I20260812 08:03:55.221179 22218 external_mini_cluster.cc:1372] Running /tmp/dist-test-taskijWYiT/build/debug/bin/kudu
/tmp/dist-test-taskijWYiT/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskijWYiT/test-tmp/hybrid_clock-test.0.HybridClockTest.SlowClockInitialisation.1786521827826957-22218-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskijWYiT/test-tmp/hybrid_clock-test.0.HybridClockTest.SlowClockInitialisation.1786521827826957-22218-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskijWYiT/test-tmp/hybrid_clock-test.0.HybridClockTest.SlowClockInitialisation.1786521827826957-22218-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskijWYiT/test-tmp/hybrid_clock-test.0.HybridClockTest.SlowClockInitialisation.1786521827826957-22218-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.21.178.129:44811
--local_ip_for_outbound_sockets=127.21.178.129
--tserver_master_addrs=127.21.178.190:34091,127.21.178.189:33967,127.21.178.188:36361
--webserver_port=46079
--webserver_interface=127.21.178.129
--builtin_ntp_servers=127.21.178.148:45861
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--hybrid_clock_inject_init_delay_ms=100
--wall_clock_jump_detection=enabled with env {}
I20260812 08:03:55.225461 23411 sys_catalog.cc:263] Verifying existing consensus state
I20260812 08:03:55.227244 23411 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fe980ff120154749b98f3e31aba0cdd6: Bootstrap starting.
I20260812 08:03:55.241294 23411 log.cc:826] T 00000000000000000000000000000000 P fe980ff120154749b98f3e31aba0cdd6: Log is configured to *not* fsync() on all Append() calls
I20260812 08:03:55.247891 23411 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fe980ff120154749b98f3e31aba0cdd6: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 08:03:55.248394 23411 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fe980ff120154749b98f3e31aba0cdd6: Bootstrap complete.
I20260812 08:03:55.251987 23411 raft_consensus.cc:359] T 00000000000000000000000000000000 P fe980ff120154749b98f3e31aba0cdd6 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43d0f953cc1d438db131bdf6e1ac6c78" member_type: VOTER last_known_addr { host: "127.21.178.190" port: 34091 } } peers { permanent_uuid: "6d7aeaee8b1c46d29a54da6cc92cbf3d" member_type: VOTER last_known_addr { host: "127.21.178.189" port: 33967 } } peers { permanent_uuid: "fe980ff120154749b98f3e31aba0cdd6" member_type: VOTER last_known_addr { host: "127.21.178.188" port: 36361 } }
I20260812 08:03:55.252470 23411 raft_consensus.cc:740] T 00000000000000000000000000000000 P fe980ff120154749b98f3e31aba0cdd6 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: fe980ff120154749b98f3e31aba0cdd6, State: Initialized, Role: FOLLOWER
I20260812 08:03:55.253067 23411 consensus_queue.cc:260] T 00000000000000000000000000000000 P fe980ff120154749b98f3e31aba0cdd6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43d0f953cc1d438db131bdf6e1ac6c78" member_type: VOTER last_known_addr { host: "127.21.178.190" port: 34091 } } peers { permanent_uuid: "6d7aeaee8b1c46d29a54da6cc92cbf3d" member_type: VOTER last_known_addr { host: "127.21.178.189" port: 33967 } } peers { permanent_uuid: "fe980ff120154749b98f3e31aba0cdd6" member_type: VOTER last_known_addr { host: "127.21.178.188" port: 36361 } }
I20260812 08:03:55.253572 23415 sys_catalog.cc:455] T 00000000000000000000000000000000 P fe980ff120154749b98f3e31aba0cdd6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43d0f953cc1d438db131bdf6e1ac6c78" member_type: VOTER last_known_addr { host: "127.21.178.190" port: 34091 } } peers { permanent_uuid: "6d7aeaee8b1c46d29a54da6cc92cbf3d" member_type: VOTER last_known_addr { host: "127.21.178.189" port: 33967 } } peers { permanent_uuid: "fe980ff120154749b98f3e31aba0cdd6" member_type: VOTER last_known_addr { host: "127.21.178.188" port: 36361 } } }
I20260812 08:03:55.253744 23415 sys_catalog.cc:458] T 00000000000000000000000000000000 P fe980ff120154749b98f3e31aba0cdd6 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 08:03:55.254299 23411 sys_catalog.cc:565] T 00000000000000000000000000000000 P fe980ff120154749b98f3e31aba0cdd6 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 08:03:55.259629 23430 catalog_manager.cc:1295] Loaded cluster ID: cf68c36522494160a7ebf067d47f34e3
I20260812 08:03:55.259718 23430 catalog_manager.cc:1588] T 00000000000000000000000000000000 P fe980ff120154749b98f3e31aba0cdd6: loading cluster ID for follower catalog manager: success
I20260812 08:03:55.261523 23430 catalog_manager.cc:1610] T 00000000000000000000000000000000 P fe980ff120154749b98f3e31aba0cdd6: acquiring CA information for follower catalog manager: success
I20260812 08:03:55.262934 23430 catalog_manager.cc:1638] T 00000000000000000000000000000000 P fe980ff120154749b98f3e31aba0cdd6: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260812 08:03:55.294710 23321 master_runner.cc:264] No leader master found from master fe980ff120154749b98f3e31aba0cdd6
I20260812 08:03:55.294807 23321 master_runner.cc:419] Couldn't verify the masters in the cluster. Trying again...
W20260812 08:03:55.366230 23413 flags.cc:432] Enabled unsafe flag: --hybrid_clock_inject_init_delay_ms=100
W20260812 08:03:55.366580 23413 flags.cc:432] Enabled unsafe flag: --openssl_security_level_override=0
W20260812 08:03:55.366636 23413 flags.cc:432] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260812 08:03:55.366721 23413 flags.cc:432] Enabled unsafe flag: --never_fsync=true
W20260812 08:03:55.370649 23413 flags.cc:432] Enabled experimental flag: --ipki_server_key_size=768
W20260812 08:03:55.370776 23413 flags.cc:432] Enabled experimental flag: --wall_clock_jump_detection=enabled
W20260812 08:03:55.370900 23413 flags.cc:432] Enabled experimental flag: --local_ip_for_outbound_sockets=127.21.178.129
I20260812 08:03:55.375465 23413 tablet_server_runner.cc:78] Tablet server non-default flags:
--log_dir=/tmp/dist-test-taskijWYiT/test-tmp/hybrid_clock-test.0.HybridClockTest.SlowClockInitialisation.1786521827826957-22218-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.21.178.148:45861
--hybrid_clock_inject_init_delay_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskijWYiT/test-tmp/hybrid_clock-test.0.HybridClockTest.SlowClockInitialisation.1786521827826957-22218-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskijWYiT/test-tmp/hybrid_clock-test.0.HybridClockTest.SlowClockInitialisation.1786521827826957-22218-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.21.178.129:44811
--rpc_server_allow_ephemeral_ports=true
--wall_clock_jump_detection=enabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskijWYiT/test-tmp/hybrid_clock-test.0.HybridClockTest.SlowClockInitialisation.1786521827826957-22218-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.21.178.129
--webserver_port=46079
--tserver_master_addrs=127.21.178.190:34091,127.21.178.189:33967,127.21.178.188:36361
--never_fsync=true
--heap_profile_path=/tmp/kudu.23413
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.21.178.129

Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 70c51312f1186a8ae9f46647160b7730123f7601-dirty
build type DEBUG
built by None at 12 Aug 2026 06:06:26 UTC on 39be9dd7121a
I20260812 08:03:55.376820 23413 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 08:03:55.378055 23413 file_cache.cc:504] Constructed file cache file cache with capacity 419430
I20260812 08:03:55.382424 23413 server_base.cc:839] enabling wall clock jump detection
I20260812 08:03:55.382704 23413 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
W20260812 08:03:55.382879 23413 hybrid_clock.cc:606] Injecting 100ms delay in HybridClock initialization process
I20260812 08:03:55.484305 23413 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260812 08:03:55.485523 23413 hybrid_clock.cc:648] HybridClock initialized: now 1786521835485502 us; error 48 us; skew 500 ppm
I20260812 08:03:55.487882 23413 webserver.cc:533] Webserver started at http://127.21.178.129:46079/ using document root <none> and password file <none>
I20260812 08:03:55.488662 23413 fs_manager.cc:362] Metadata directory not provided
I20260812 08:03:55.488768 23413 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 08:03:55.492800 23413 fs_manager.cc:714] Time spent opening directory manager: real 0.003s	user 0.003s	sys 0.001s
I20260812 08:03:55.495072 23448 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:03:55.496409 23413 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
I20260812 08:03:55.496534 23413 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskijWYiT/test-tmp/hybrid_clock-test.0.HybridClockTest.SlowClockInitialisation.1786521827826957-22218-0/minicluster-data/ts-0/data,/tmp/dist-test-taskijWYiT/test-tmp/hybrid_clock-test.0.HybridClockTest.SlowClockInitialisation.1786521827826957-22218-0/minicluster-data/ts-0/wal
uuid: "6ecd0a55071d44968e2769ebe34f0036"
format_stamp: "Formatted at 2026-08-12 08:03:51 on dist-test-slave-8w9v"
I20260812 08:03:55.496968 23413 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskijWYiT/test-tmp/hybrid_clock-test.0.HybridClockTest.SlowClockInitialisation.1786521827826957-22218-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskijWYiT/test-tmp/hybrid_clock-test.0.HybridClockTest.SlowClockInitialisation.1786521827826957-22218-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskijWYiT/test-tmp/hybrid_clock-test.0.HybridClockTest.SlowClockInitialisation.1786521827826957-22218-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 08:03:55.509020 23413 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:03:55.509799 23413 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 08:03:55.510007 23413 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:03:55.510687 23413 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260812 08:03:55.511876 23413 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 08:03:55.511935 23413 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:03:55.511987 23413 ts_tablet_manager.cc:616] Registered 0 tablets
I20260812 08:03:55.512015 23413 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:03:55.523826 23413 rpc_server.cc:307] RPC server started. Bound to: 127.21.178.129:44811
I20260812 08:03:55.523937 23580 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.21.178.129:44811 every 8 connection(s)
I20260812 08:03:55.525194 23413 server_base.cc:1193] Dumped server information to /tmp/dist-test-taskijWYiT/test-tmp/hybrid_clock-test.0.HybridClockTest.SlowClockInitialisation.1786521827826957-22218-0/minicluster-data/ts-0/data/info.pb
I20260812 08:03:55.531176 22218 external_mini_cluster.cc:1434] Started /tmp/dist-test-taskijWYiT/build/debug/bin/kudu as pid 23413
I20260812 08:03:55.543805 23583 heartbeater.cc:344] Connected to a master server at 127.21.178.189:33967
I20260812 08:03:55.543807 23581 heartbeater.cc:344] Connected to a master server at 127.21.178.188:36361
I20260812 08:03:55.543915 23582 heartbeater.cc:344] Connected to a master server at 127.21.178.190:34091
I20260812 08:03:55.544144 23583 heartbeater.cc:461] Registering TS with master...
I20260812 08:03:55.544201 23582 heartbeater.cc:461] Registering TS with master...
I20260812 08:03:55.544142 23581 heartbeater.cc:461] Registering TS with master...
I20260812 08:03:55.544888 23583 heartbeater.cc:507] Master 127.21.178.189:33967 requested a full tablet report, sending...
I20260812 08:03:55.545240 23581 heartbeater.cc:507] Master 127.21.178.188:36361 requested a full tablet report, sending...
I20260812 08:03:55.544888 23582 heartbeater.cc:507] Master 127.21.178.190:34091 requested a full tablet report, sending...
I20260812 08:03:55.546731 23366 ts_manager.cc:194] Registered new tserver with Master: 6ecd0a55071d44968e2769ebe34f0036 (127.21.178.129:44811)
I20260812 08:03:55.546733 23275 ts_manager.cc:194] Registered new tserver with Master: 6ecd0a55071d44968e2769ebe34f0036 (127.21.178.129:44811)
I20260812 08:03:55.547691 23183 ts_manager.cc:194] Registered new tserver with Master: 6ecd0a55071d44968e2769ebe34f0036 (127.21.178.129:44811)
I20260812 08:03:55.558153 22218 external_mini_cluster.cc:955] 1 TS(s) registered with all masters
I20260812 08:03:55.953073 23590 raft_consensus.cc:493] T 00000000000000000000000000000000 P 43d0f953cc1d438db131bdf6e1ac6c78 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 08:03:55.953305 23590 raft_consensus.cc:515] T 00000000000000000000000000000000 P 43d0f953cc1d438db131bdf6e1ac6c78 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43d0f953cc1d438db131bdf6e1ac6c78" member_type: VOTER last_known_addr { host: "127.21.178.190" port: 34091 } } peers { permanent_uuid: "6d7aeaee8b1c46d29a54da6cc92cbf3d" member_type: VOTER last_known_addr { host: "127.21.178.189" port: 33967 } } peers { permanent_uuid: "fe980ff120154749b98f3e31aba0cdd6" member_type: VOTER last_known_addr { host: "127.21.178.188" port: 36361 } }
I20260812 08:03:55.954610 23590 leader_election.cc:290] T 00000000000000000000000000000000 P 43d0f953cc1d438db131bdf6e1ac6c78 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6d7aeaee8b1c46d29a54da6cc92cbf3d (127.21.178.189:33967), fe980ff120154749b98f3e31aba0cdd6 (127.21.178.188:36361)
I20260812 08:03:55.959022 23378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "43d0f953cc1d438db131bdf6e1ac6c78" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "fe980ff120154749b98f3e31aba0cdd6" is_pre_election: true
I20260812 08:03:55.959409 23378 raft_consensus.cc:2468] T 00000000000000000000000000000000 P fe980ff120154749b98f3e31aba0cdd6 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 43d0f953cc1d438db131bdf6e1ac6c78 in term 2.
I20260812 08:03:55.959362 23286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "43d0f953cc1d438db131bdf6e1ac6c78" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6d7aeaee8b1c46d29a54da6cc92cbf3d" is_pre_election: true
I20260812 08:03:55.959650 23286 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 6d7aeaee8b1c46d29a54da6cc92cbf3d [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 43d0f953cc1d438db131bdf6e1ac6c78 in term 2.
I20260812 08:03:55.959945 23166 leader_election.cc:304] T 00000000000000000000000000000000 P 43d0f953cc1d438db131bdf6e1ac6c78 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 43d0f953cc1d438db131bdf6e1ac6c78, fe980ff120154749b98f3e31aba0cdd6; no voters: 
I20260812 08:03:55.960253 23590 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 43d0f953cc1d438db131bdf6e1ac6c78 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260812 08:03:55.960331 23590 raft_consensus.cc:493] T 00000000000000000000000000000000 P 43d0f953cc1d438db131bdf6e1ac6c78 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 08:03:55.960374 23590 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 43d0f953cc1d438db131bdf6e1ac6c78 [term 2 FOLLOWER]: Advancing to term 3
I20260812 08:03:55.962319 23590 raft_consensus.cc:515] T 00000000000000000000000000000000 P 43d0f953cc1d438db131bdf6e1ac6c78 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43d0f953cc1d438db131bdf6e1ac6c78" member_type: VOTER last_known_addr { host: "127.21.178.190" port: 34091 } } peers { permanent_uuid: "6d7aeaee8b1c46d29a54da6cc92cbf3d" member_type: VOTER last_known_addr { host: "127.21.178.189" port: 33967 } } peers { permanent_uuid: "fe980ff120154749b98f3e31aba0cdd6" member_type: VOTER last_known_addr { host: "127.21.178.188" port: 36361 } }
I20260812 08:03:55.962802 23590 leader_election.cc:290] T 00000000000000000000000000000000 P 43d0f953cc1d438db131bdf6e1ac6c78 [CANDIDATE]: Term 3 election: Requested vote from peers 6d7aeaee8b1c46d29a54da6cc92cbf3d (127.21.178.189:33967), fe980ff120154749b98f3e31aba0cdd6 (127.21.178.188:36361)
I20260812 08:03:55.963207 23286 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "43d0f953cc1d438db131bdf6e1ac6c78" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6d7aeaee8b1c46d29a54da6cc92cbf3d"
I20260812 08:03:55.963346 23286 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 6d7aeaee8b1c46d29a54da6cc92cbf3d [term 2 FOLLOWER]: Advancing to term 3
I20260812 08:03:55.963449 23378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "43d0f953cc1d438db131bdf6e1ac6c78" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "fe980ff120154749b98f3e31aba0cdd6"
I20260812 08:03:55.963611 23378 raft_consensus.cc:3060] T 00000000000000000000000000000000 P fe980ff120154749b98f3e31aba0cdd6 [term 2 FOLLOWER]: Advancing to term 3
I20260812 08:03:55.965265 23286 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 6d7aeaee8b1c46d29a54da6cc92cbf3d [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 43d0f953cc1d438db131bdf6e1ac6c78 in term 3.
I20260812 08:03:55.965677 23165 leader_election.cc:304] T 00000000000000000000000000000000 P 43d0f953cc1d438db131bdf6e1ac6c78 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 43d0f953cc1d438db131bdf6e1ac6c78, 6d7aeaee8b1c46d29a54da6cc92cbf3d; no voters: 
I20260812 08:03:55.965705 23378 raft_consensus.cc:2468] T 00000000000000000000000000000000 P fe980ff120154749b98f3e31aba0cdd6 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 43d0f953cc1d438db131bdf6e1ac6c78 in term 3.
I20260812 08:03:55.965883 23590 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 43d0f953cc1d438db131bdf6e1ac6c78 [term 3 FOLLOWER]: Leader election won for term 3
I20260812 08:03:55.966099 23590 raft_consensus.cc:697] T 00000000000000000000000000000000 P 43d0f953cc1d438db131bdf6e1ac6c78 [term 3 LEADER]: Becoming Leader. State: Replica: 43d0f953cc1d438db131bdf6e1ac6c78, State: Running, Role: LEADER
I20260812 08:03:55.966427 23590 consensus_queue.cc:237] T 00000000000000000000000000000000 P 43d0f953cc1d438db131bdf6e1ac6c78 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43d0f953cc1d438db131bdf6e1ac6c78" member_type: VOTER last_known_addr { host: "127.21.178.190" port: 34091 } } peers { permanent_uuid: "6d7aeaee8b1c46d29a54da6cc92cbf3d" member_type: VOTER last_known_addr { host: "127.21.178.189" port: 33967 } } peers { permanent_uuid: "fe980ff120154749b98f3e31aba0cdd6" member_type: VOTER last_known_addr { host: "127.21.178.188" port: 36361 } }
I20260812 08:03:55.968286 23593 sys_catalog.cc:455] T 00000000000000000000000000000000 P 43d0f953cc1d438db131bdf6e1ac6c78 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 43d0f953cc1d438db131bdf6e1ac6c78. Latest consensus state: current_term: 3 leader_uuid: "43d0f953cc1d438db131bdf6e1ac6c78" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43d0f953cc1d438db131bdf6e1ac6c78" member_type: VOTER last_known_addr { host: "127.21.178.190" port: 34091 } } peers { permanent_uuid: "6d7aeaee8b1c46d29a54da6cc92cbf3d" member_type: VOTER last_known_addr { host: "127.21.178.189" port: 33967 } } peers { permanent_uuid: "fe980ff120154749b98f3e31aba0cdd6" member_type: VOTER last_known_addr { host: "127.21.178.188" port: 36361 } } }
I20260812 08:03:55.968410 23593 sys_catalog.cc:458] T 00000000000000000000000000000000 P 43d0f953cc1d438db131bdf6e1ac6c78 [sys.catalog]: This master's current role is: LEADER
I20260812 08:03:55.968736 23596 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260812 08:03:55.969755 23596 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260812 08:03:55.970156 23596 catalog_manager.cc:1295] Loaded cluster ID: cf68c36522494160a7ebf067d47f34e3
I20260812 08:03:55.970230 23596 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260812 08:03:55.970906 23596 catalog_manager.cc:1540] Loading token signing keys...
I20260812 08:03:55.971336 23596 catalog_manager.cc:6103] T 00000000000000000000000000000000 P 43d0f953cc1d438db131bdf6e1ac6c78: Loaded TSK: 0
I20260812 08:03:55.971978 23596 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260812 08:03:56.550333 23183 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.21.178.129:38255
I20260812 08:03:56.556787 23286 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 6d7aeaee8b1c46d29a54da6cc92cbf3d [term 3 FOLLOWER]: Refusing update from remote peer 43d0f953cc1d438db131bdf6e1ac6c78: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260812 08:03:56.557566 23598 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 43d0f953cc1d438db131bdf6e1ac6c78 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6d7aeaee8b1c46d29a54da6cc92cbf3d" member_type: VOTER last_known_addr { host: "127.21.178.189" port: 33967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
I20260812 08:03:56.559948 23600 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6d7aeaee8b1c46d29a54da6cc92cbf3d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 43d0f953cc1d438db131bdf6e1ac6c78. Latest consensus state: current_term: 3 leader_uuid: "43d0f953cc1d438db131bdf6e1ac6c78" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43d0f953cc1d438db131bdf6e1ac6c78" member_type: VOTER last_known_addr { host: "127.21.178.190" port: 34091 } } peers { permanent_uuid: "6d7aeaee8b1c46d29a54da6cc92cbf3d" member_type: VOTER last_known_addr { host: "127.21.178.189" port: 33967 } } peers { permanent_uuid: "fe980ff120154749b98f3e31aba0cdd6" member_type: VOTER last_known_addr { host: "127.21.178.188" port: 36361 } } }
I20260812 08:03:56.560179 23600 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6d7aeaee8b1c46d29a54da6cc92cbf3d [sys.catalog]: This master's current role is: FOLLOWER
I20260812 08:03:56.561810 23604 sys_catalog.cc:455] T 00000000000000000000000000000000 P 43d0f953cc1d438db131bdf6e1ac6c78 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "43d0f953cc1d438db131bdf6e1ac6c78" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43d0f953cc1d438db131bdf6e1ac6c78" member_type: VOTER last_known_addr { host: "127.21.178.190" port: 34091 } } peers { permanent_uuid: "6d7aeaee8b1c46d29a54da6cc92cbf3d" member_type: VOTER last_known_addr { host: "127.21.178.189" port: 33967 } } peers { permanent_uuid: "fe980ff120154749b98f3e31aba0cdd6" member_type: VOTER last_known_addr { host: "127.21.178.188" port: 36361 } } }
I20260812 08:03:56.562176 23600 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6d7aeaee8b1c46d29a54da6cc92cbf3d [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "43d0f953cc1d438db131bdf6e1ac6c78" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43d0f953cc1d438db131bdf6e1ac6c78" member_type: VOTER last_known_addr { host: "127.21.178.190" port: 34091 } } peers { permanent_uuid: "6d7aeaee8b1c46d29a54da6cc92cbf3d" member_type: VOTER last_known_addr { host: "127.21.178.189" port: 33967 } } peers { permanent_uuid: "fe980ff120154749b98f3e31aba0cdd6" member_type: VOTER last_known_addr { host: "127.21.178.188" port: 36361 } } }
I20260812 08:03:56.562273 23600 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6d7aeaee8b1c46d29a54da6cc92cbf3d [sys.catalog]: This master's current role is: FOLLOWER
I20260812 08:03:56.562345 23600 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6d7aeaee8b1c46d29a54da6cc92cbf3d [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "43d0f953cc1d438db131bdf6e1ac6c78" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43d0f953cc1d438db131bdf6e1ac6c78" member_type: VOTER last_known_addr { host: "127.21.178.190" port: 34091 } } peers { permanent_uuid: "6d7aeaee8b1c46d29a54da6cc92cbf3d" member_type: VOTER last_known_addr { host: "127.21.178.189" port: 33967 } } peers { permanent_uuid: "fe980ff120154749b98f3e31aba0cdd6" member_type: VOTER last_known_addr { host: "127.21.178.188" port: 36361 } } }
I20260812 08:03:56.562402 23600 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6d7aeaee8b1c46d29a54da6cc92cbf3d [sys.catalog]: This master's current role is: FOLLOWER
I20260812 08:03:56.562563 23604 sys_catalog.cc:458] T 00000000000000000000000000000000 P 43d0f953cc1d438db131bdf6e1ac6c78 [sys.catalog]: This master's current role is: LEADER
I20260812 08:03:56.562788 23598 sys_catalog.cc:455] T 00000000000000000000000000000000 P 43d0f953cc1d438db131bdf6e1ac6c78 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "43d0f953cc1d438db131bdf6e1ac6c78" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43d0f953cc1d438db131bdf6e1ac6c78" member_type: VOTER last_known_addr { host: "127.21.178.190" port: 34091 } } peers { permanent_uuid: "6d7aeaee8b1c46d29a54da6cc92cbf3d" member_type: VOTER last_known_addr { host: "127.21.178.189" port: 33967 } } peers { permanent_uuid: "fe980ff120154749b98f3e31aba0cdd6" member_type: VOTER last_known_addr { host: "127.21.178.188" port: 36361 } } }
I20260812 08:03:56.562901 23598 sys_catalog.cc:458] T 00000000000000000000000000000000 P 43d0f953cc1d438db131bdf6e1ac6c78 [sys.catalog]: This master's current role is: LEADER
I20260812 08:03:56.563868 23378 raft_consensus.cc:1275] T 00000000000000000000000000000000 P fe980ff120154749b98f3e31aba0cdd6 [term 3 FOLLOWER]: Refusing update from remote peer 43d0f953cc1d438db131bdf6e1ac6c78: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260812 08:03:56.564491 23609 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 43d0f953cc1d438db131bdf6e1ac6c78 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fe980ff120154749b98f3e31aba0cdd6" member_type: VOTER last_known_addr { host: "127.21.178.188" port: 36361 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
I20260812 08:03:56.567497 23613 sys_catalog.cc:455] T 00000000000000000000000000000000 P fe980ff120154749b98f3e31aba0cdd6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 43d0f953cc1d438db131bdf6e1ac6c78. Latest consensus state: current_term: 3 leader_uuid: "43d0f953cc1d438db131bdf6e1ac6c78" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43d0f953cc1d438db131bdf6e1ac6c78" member_type: VOTER last_known_addr { host: "127.21.178.190" port: 34091 } } peers { permanent_uuid: "6d7aeaee8b1c46d29a54da6cc92cbf3d" member_type: VOTER last_known_addr { host: "127.21.178.189" port: 33967 } } peers { permanent_uuid: "fe980ff120154749b98f3e31aba0cdd6" member_type: VOTER last_known_addr { host: "127.21.178.188" port: 36361 } } }
I20260812 08:03:56.567631 23613 sys_catalog.cc:458] T 00000000000000000000000000000000 P fe980ff120154749b98f3e31aba0cdd6 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 08:03:56.567713 23613 sys_catalog.cc:455] T 00000000000000000000000000000000 P fe980ff120154749b98f3e31aba0cdd6 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "43d0f953cc1d438db131bdf6e1ac6c78" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43d0f953cc1d438db131bdf6e1ac6c78" member_type: VOTER last_known_addr { host: "127.21.178.190" port: 34091 } } peers { permanent_uuid: "6d7aeaee8b1c46d29a54da6cc92cbf3d" member_type: VOTER last_known_addr { host: "127.21.178.189" port: 33967 } } peers { permanent_uuid: "fe980ff120154749b98f3e31aba0cdd6" member_type: VOTER last_known_addr { host: "127.21.178.188" port: 36361 } } }
I20260812 08:03:56.567771 23613 sys_catalog.cc:458] T 00000000000000000000000000000000 P fe980ff120154749b98f3e31aba0cdd6 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 08:03:56.567819 23613 sys_catalog.cc:455] T 00000000000000000000000000000000 P fe980ff120154749b98f3e31aba0cdd6 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "43d0f953cc1d438db131bdf6e1ac6c78" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43d0f953cc1d438db131bdf6e1ac6c78" member_type: VOTER last_known_addr { host: "127.21.178.190" port: 34091 } } peers { permanent_uuid: "6d7aeaee8b1c46d29a54da6cc92cbf3d" member_type: VOTER last_known_addr { host: "127.21.178.189" port: 33967 } } peers { permanent_uuid: "fe980ff120154749b98f3e31aba0cdd6" member_type: VOTER last_known_addr { host: "127.21.178.188" port: 36361 } } }
I20260812 08:03:56.567871 23613 sys_catalog.cc:458] T 00000000000000000000000000000000 P fe980ff120154749b98f3e31aba0cdd6 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 08:03:56.598156 22218 webui_checker.cc:88] shutting down CURL thread
I20260812 08:03:56.603075 22218 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskijWYiT/build/debug/bin/kudu with pid 23413
I20260812 08:03:56.612895 22218 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskijWYiT/build/debug/bin/kudu with pid 23148
I20260812 08:03:56.622906 22218 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskijWYiT/build/debug/bin/kudu with pid 23229
I20260812 08:03:56.632668 22218 external_mini_cluster.cc:1664] Killing /tmp/dist-test-taskijWYiT/build/debug/bin/kudu with pid 23321
2026-08-12T08:03:56Z chronyd exiting
[       OK ] HybridClockTest.SlowClockInitialisation (6798 ms)
[ RUN      ] HybridClockTest.TimeSourceAutoSelection
I20260812 08:03:56.658938 22218 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260812 08:03:56.658998 22218 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260812 08:03:56.659014 22218 hybrid_clock.cc:648] HybridClock initialized: now 1786521836659014 us; error 0 us; skew 500 ppm
W20260812 08:03:56.660876 23622 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:03:56.661044 23626 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:03:56.661186 23623 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:03:56.661473 22218 hybrid_clock.cc:561] auto-selected time source 'builtin' for the hybrid clock
I20260812 08:03:56.661527 22218 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260812 08:03:56.662016 22218 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=60 seconds for the clock to synchronize
I20260812 08:03:56.663182 22218 hybrid_clock.cc:648] HybridClock initialized: now 1786521836663120 us; error 197 us; skew 500 ppm
W20260812 08:03:56.663400 22218 builtin_ntp.cc:688] coult not shutdown socket: Network error: shutdown error: Transport endpoint is not connected (error 107)
[       OK ] HybridClockTest.TimeSourceAutoSelection (5 ms)
[ RUN      ] HybridClockTest.AutoTimeSourceNoDedicatedNtpServer
I20260812 08:03:56.664728 22218 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260812 08:03:56.664768 22218 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260812 08:03:56.664783 22218 hybrid_clock.cc:648] HybridClock initialized: now 1786521836664783 us; error 0 us; skew 500 ppm
W20260812 08:03:56.665041 23629 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: An unknown option was passed in to libcurl
W20260812 08:03:56.665091 23630 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: An unknown option was passed in to libcurl
W20260812 08:03:56.665169 23632 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: An unknown option was passed in to libcurl
W20260812 08:03:56.665295 23631 instance_detector.cc:116] could not retrieve GCE instance metadata: Network error: curl error: An unknown option was passed in to libcurl
W20260812 08:03:56.665519 23633 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: An unknown option was passed in to libcurl
W20260812 08:03:56.665576 23634 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: An unknown option was passed in to libcurl
W20260812 08:03:56.665664 23635 instance_detector.cc:116] could not retrieve GCE instance metadata: Network error: curl error: An unknown option was passed in to libcurl
W20260812 08:03:56.665696 23638 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: An unknown option was passed in to libcurl
I20260812 08:03:56.665853 22218 hybrid_clock.cc:561] auto-selected time source 'builtin' for the hybrid clock
W20260812 08:03:56.666054 23639 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: An unknown option was passed in to libcurl
W20260812 08:03:56.666127 23640 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: An unknown option was passed in to libcurl
W20260812 08:03:56.666220 23641 instance_detector.cc:116] could not retrieve GCE instance metadata: Network error: curl error: An unknown option was passed in to libcurl
W20260812 08:03:56.666280 23642 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: An unknown option was passed in to libcurl
I20260812 08:03:56.666420 22218 hybrid_clock.cc:561] auto-selected time source 'system' for the hybrid clock
W20260812 08:03:56.666601 23643 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: An unknown option was passed in to libcurl
W20260812 08:03:56.666690 23644 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: An unknown option was passed in to libcurl
W20260812 08:03:56.666769 23646 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: An unknown option was passed in to libcurl
W20260812 08:03:56.666791 23645 instance_detector.cc:116] could not retrieve GCE instance metadata: Network error: curl error: An unknown option was passed in to libcurl
I20260812 08:03:56.666905 22218 hybrid_clock.cc:561] auto-selected time source 'system' for the hybrid clock
[       OK ] HybridClockTest.AutoTimeSourceNoDedicatedNtpServer (3 ms)
[ RUN      ] HybridClockTest.ClockJumpDetection
I20260812 08:03:56.667960 22218 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260812 08:03:56.668001 22218 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260812 08:03:56.668015 22218 hybrid_clock.cc:648] HybridClock initialized: now 1786521836668016 us; error 0 us; skew 500 ppm
I20260812 08:03:56.668111 22218 hybrid_clock.cc:584] initializing the hybrid clock with 'mock' time source
I20260812 08:03:56.668170 22218 hybrid_clock.cc:648] HybridClock initialized: now 0 us; error 0 us; skew 500 ppm
[       OK ] HybridClockTest.ClockJumpDetection (1 ms)
[ RUN      ] HybridClockTest.TestNtpDiagnostics
I20260812 08:03:56.669277 22218 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260812 08:03:56.669337 22218 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260812 08:03:56.669363 22218 hybrid_clock.cc:648] HybridClock initialized: now 1786521836669361 us; error 0 us; skew 500 ppm
I20260812 08:03:56.669447 22218 hybrid_clock.cc:584] initializing the hybrid clock with 'system' time source
I20260812 08:03:56.669646 22218 hybrid_clock.cc:648] HybridClock initialized: now 1786521836669643 us; error 20920 us; skew 500 ppm
[       OK ] HybridClockTest.TestNtpDiagnostics (3071 ms)
[----------] 13 tests from HybridClockTest (11892 ms total)

[----------] 2 tests from Perf/HybridClockJumpProtectionTest
[ RUN      ] Perf/HybridClockJumpProtectionTest.BasicPerf/0
I20260812 08:03:59.740857 22218 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260812 08:03:59.740929 22218 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260812 08:03:59.740952 22218 hybrid_clock.cc:648] HybridClock initialized: now 1786521839740951 us; error 0 us; skew 500 ppm
I20260812 08:04:00.475660 22218 hybrid_clock-test.cc:697] 1000000 iterations in 734.578397 ms
[       OK ] Perf/HybridClockJumpProtectionTest.BasicPerf/0 (737 ms)
[ RUN      ] Perf/HybridClockJumpProtectionTest.BasicPerf/1
I20260812 08:04:00.478987 22218 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260812 08:04:00.479069 22218 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260812 08:04:00.479094 22218 hybrid_clock.cc:648] HybridClock initialized: now 1786521840479093 us; error 0 us; skew 500 ppm
I20260812 08:04:01.247299 22218 hybrid_clock-test.cc:697] 1000000 iterations in 768.085617 ms
[       OK ] Perf/HybridClockJumpProtectionTest.BasicPerf/1 (769 ms)
[----------] 2 tests from Perf/HybridClockJumpProtectionTest (1507 ms total)

[----------] Global test environment tear-down
[==========] 17 tests from 3 test suites ran. (13412 ms total)
[  PASSED  ] 17 tests.
