[==========] Running 6 tests from 3 test suites.
[----------] Global test environment set-up.
[----------] 3 tests from PrometheusSDITest
[ RUN      ] PrometheusSDITest.BasicSDAndScrape
WARNING: Logging before InitGoogleLogging() is written to STDERR
I20260812 08:02:50.788453 19841 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.19.96.126:43045
I20260812 08:02:50.789536 19841 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260812 08:02:50.790150 19841 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 08:02:50.796870 19848 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:02:50.796941 19851 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:02:50.797111 19849 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:02:50.797016 19841 server_base.cc:1061] running on GCE node
I20260812 08:02:50.797667 19841 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260812 08:02:50.797873 19841 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260812 08:02:50.797940 19841 hybrid_clock.cc:648] HybridClock initialized: now 1786521770797936 us; error 0 us; skew 500 ppm
I20260812 08:02:50.799798 19841 webserver.cc:533] Webserver started at http://127.19.96.126:36419/ using document root <none> and password file <none>
I20260812 08:02:50.800500 19841 fs_manager.cc:362] Metadata directory not provided
I20260812 08:02:50.800598 19841 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 08:02:50.800879 19841 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 08:02:50.802527 19841 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task3yToRy/test-tmp/prometheus_sd-itest.0.PrometheusSDITest.BasicSDAndScrape.1786521770778185-19841-0/minicluster-data/master-0-root/instance:
uuid: "92843689dd954d84a9f63fb483566581"
format_stamp: "Formatted at 2026-08-12 08:02:50 on dist-test-slave-8w9v"
I20260812 08:02:50.806277 19841 fs_manager.cc:696] Time spent creating directory manager: real 0.003s	user 0.005s	sys 0.000s
I20260812 08:02:50.808645 19856 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:02:50.809768 19841 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260812 08:02:50.809966 19841 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task3yToRy/test-tmp/prometheus_sd-itest.0.PrometheusSDITest.BasicSDAndScrape.1786521770778185-19841-0/minicluster-data/master-0-root
uuid: "92843689dd954d84a9f63fb483566581"
format_stamp: "Formatted at 2026-08-12 08:02:50 on dist-test-slave-8w9v"
I20260812 08:02:50.810087 19841 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task3yToRy/test-tmp/prometheus_sd-itest.0.PrometheusSDITest.BasicSDAndScrape.1786521770778185-19841-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-task3yToRy/test-tmp/prometheus_sd-itest.0.PrometheusSDITest.BasicSDAndScrape.1786521770778185-19841-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-task3yToRy/test-tmp/prometheus_sd-itest.0.PrometheusSDITest.BasicSDAndScrape.1786521770778185-19841-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 08:02:50.836561 19841 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:02:50.837275 19841 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260812 08:02:50.837463 19841 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:02:50.845691 19918 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.19.96.126:43045 every 8 connection(s)
I20260812 08:02:50.845700 19841 rpc_server.cc:307] RPC server started. Bound to: 127.19.96.126:43045
I20260812 08:02:50.849256 19919 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 08:02:50.854993 19919 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 92843689dd954d84a9f63fb483566581: Bootstrap starting.
I20260812 08:02:50.857981 19919 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 92843689dd954d84a9f63fb483566581: Neither blocks nor log segments found. Creating new log.
I20260812 08:02:50.858999 19919 log.cc:826] T 00000000000000000000000000000000 P 92843689dd954d84a9f63fb483566581: Log is configured to *not* fsync() on all Append() calls
I20260812 08:02:50.860901 19919 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 92843689dd954d84a9f63fb483566581: No bootstrap required, opened a new log
I20260812 08:02:50.863992 19919 raft_consensus.cc:359] T 00000000000000000000000000000000 P 92843689dd954d84a9f63fb483566581 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "92843689dd954d84a9f63fb483566581" member_type: VOTER }
I20260812 08:02:50.864271 19919 raft_consensus.cc:385] T 00000000000000000000000000000000 P 92843689dd954d84a9f63fb483566581 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 08:02:50.864379 19919 raft_consensus.cc:740] T 00000000000000000000000000000000 P 92843689dd954d84a9f63fb483566581 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 92843689dd954d84a9f63fb483566581, State: Initialized, Role: FOLLOWER
I20260812 08:02:50.865074 19919 consensus_queue.cc:260] T 00000000000000000000000000000000 P 92843689dd954d84a9f63fb483566581 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "92843689dd954d84a9f63fb483566581" member_type: VOTER }
I20260812 08:02:50.865255 19919 raft_consensus.cc:399] T 00000000000000000000000000000000 P 92843689dd954d84a9f63fb483566581 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 08:02:50.865355 19919 raft_consensus.cc:493] T 00000000000000000000000000000000 P 92843689dd954d84a9f63fb483566581 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 08:02:50.865522 19919 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 92843689dd954d84a9f63fb483566581 [term 0 FOLLOWER]: Advancing to term 1
I20260812 08:02:50.866431 19919 raft_consensus.cc:515] T 00000000000000000000000000000000 P 92843689dd954d84a9f63fb483566581 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "92843689dd954d84a9f63fb483566581" member_type: VOTER }
I20260812 08:02:50.866935 19919 leader_election.cc:304] T 00000000000000000000000000000000 P 92843689dd954d84a9f63fb483566581 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 92843689dd954d84a9f63fb483566581; no voters: 
I20260812 08:02:50.867339 19919 leader_election.cc:290] T 00000000000000000000000000000000 P 92843689dd954d84a9f63fb483566581 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260812 08:02:50.867548 19922 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 92843689dd954d84a9f63fb483566581 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 08:02:50.867841 19922 raft_consensus.cc:697] T 00000000000000000000000000000000 P 92843689dd954d84a9f63fb483566581 [term 1 LEADER]: Becoming Leader. State: Replica: 92843689dd954d84a9f63fb483566581, State: Running, Role: LEADER
I20260812 08:02:50.868378 19922 consensus_queue.cc:237] T 00000000000000000000000000000000 P 92843689dd954d84a9f63fb483566581 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "92843689dd954d84a9f63fb483566581" member_type: VOTER }
I20260812 08:02:50.868644 19919 sys_catalog.cc:565] T 00000000000000000000000000000000 P 92843689dd954d84a9f63fb483566581 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 08:02:50.870683 19923 sys_catalog.cc:455] T 00000000000000000000000000000000 P 92843689dd954d84a9f63fb483566581 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "92843689dd954d84a9f63fb483566581" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "92843689dd954d84a9f63fb483566581" member_type: VOTER } }
I20260812 08:02:50.870837 19923 sys_catalog.cc:458] T 00000000000000000000000000000000 P 92843689dd954d84a9f63fb483566581 [sys.catalog]: This master's current role is: LEADER
I20260812 08:02:50.870719 19924 sys_catalog.cc:455] T 00000000000000000000000000000000 P 92843689dd954d84a9f63fb483566581 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 92843689dd954d84a9f63fb483566581. Latest consensus state: current_term: 1 leader_uuid: "92843689dd954d84a9f63fb483566581" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "92843689dd954d84a9f63fb483566581" member_type: VOTER } }
I20260812 08:02:50.871093 19924 sys_catalog.cc:458] T 00000000000000000000000000000000 P 92843689dd954d84a9f63fb483566581 [sys.catalog]: This master's current role is: LEADER
I20260812 08:02:50.871208 19841 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
W20260812 08:02:50.873878 19940 catalog_manager.cc:1594] T 00000000000000000000000000000000 P 92843689dd954d84a9f63fb483566581: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 08:02:50.873960 19940 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 08:02:50.874048 19939 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260812 08:02:50.874832 19939 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260812 08:02:50.880553 19939 catalog_manager.cc:1383] Generated new cluster ID: 15e4b3d11d8f480596e005faaaf1166a
I20260812 08:02:50.880642 19939 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260812 08:02:50.904899 19939 catalog_manager.cc:1406] Generated new certificate authority record
I20260812 08:02:50.906268 19939 catalog_manager.cc:1540] Loading token signing keys...
I20260812 08:02:50.914690 19939 catalog_manager.cc:6092] T 00000000000000000000000000000000 P 92843689dd954d84a9f63fb483566581: Generated new TSK 0
I20260812 08:02:50.915552 19939 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260812 08:02:50.936285 19841 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 08:02:50.939059 19945 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:02:50.939113 19946 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:02:50.939064 19949 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:02:50.939778 19841 server_base.cc:1061] running on GCE node
I20260812 08:02:50.940022 19841 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260812 08:02:50.940063 19841 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260812 08:02:50.940079 19841 hybrid_clock.cc:648] HybridClock initialized: now 1786521770940080 us; error 0 us; skew 500 ppm
I20260812 08:02:50.941131 19841 webserver.cc:533] Webserver started at http://127.19.96.65:34427/ using document root <none> and password file <none>
I20260812 08:02:50.941346 19841 fs_manager.cc:362] Metadata directory not provided
I20260812 08:02:50.941397 19841 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 08:02:50.941506 19841 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 08:02:50.941912 19841 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task3yToRy/test-tmp/prometheus_sd-itest.0.PrometheusSDITest.BasicSDAndScrape.1786521770778185-19841-0/minicluster-data/ts-0-root/instance:
uuid: "d4e468bb91134582b610f4b41b3dff74"
format_stamp: "Formatted at 2026-08-12 08:02:50 on dist-test-slave-8w9v"
I20260812 08:02:50.943608 19841 fs_manager.cc:696] Time spent creating directory manager: real 0.001s	user 0.000s	sys 0.002s
I20260812 08:02:50.944713 19956 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:02:50.944999 19841 fs_manager.cc:730] Time spent opening block manager: real 0.001s	user 0.000s	sys 0.000s
I20260812 08:02:50.945097 19841 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task3yToRy/test-tmp/prometheus_sd-itest.0.PrometheusSDITest.BasicSDAndScrape.1786521770778185-19841-0/minicluster-data/ts-0-root
uuid: "d4e468bb91134582b610f4b41b3dff74"
format_stamp: "Formatted at 2026-08-12 08:02:50 on dist-test-slave-8w9v"
I20260812 08:02:50.945179 19841 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task3yToRy/test-tmp/prometheus_sd-itest.0.PrometheusSDITest.BasicSDAndScrape.1786521770778185-19841-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-task3yToRy/test-tmp/prometheus_sd-itest.0.PrometheusSDITest.BasicSDAndScrape.1786521770778185-19841-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-task3yToRy/test-tmp/prometheus_sd-itest.0.PrometheusSDITest.BasicSDAndScrape.1786521770778185-19841-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 08:02:50.957307 19841 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:02:50.957788 19841 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:02:50.958340 19841 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260812 08:02:50.959208 19841 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 08:02:50.959257 19841 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:02:50.959340 19841 ts_tablet_manager.cc:616] Registered 0 tablets
I20260812 08:02:50.959381 19841 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:02:50.967890 19841 rpc_server.cc:307] RPC server started. Bound to: 127.19.96.65:42561
I20260812 08:02:50.967926 20030 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.19.96.65:42561 every 8 connection(s)
I20260812 08:02:50.982491 20033 heartbeater.cc:344] Connected to a master server at 127.19.96.126:43045
I20260812 08:02:50.982847 20033 heartbeater.cc:461] Registering TS with master...
I20260812 08:02:50.983376 20033 heartbeater.cc:507] Master 127.19.96.126:43045 requested a full tablet report, sending...
I20260812 08:02:50.984925 19875 ts_manager.cc:194] Registered new tserver with Master: d4e468bb91134582b610f4b41b3dff74 (127.19.96.65:42561)
I20260812 08:02:50.985608 19841 internal_mini_cluster.cc:371] 1 TS(s) registered with all masters after 0.016890887s
I20260812 08:02:50.986297 19841 internal_mini_cluster.cc:371] 1 TS(s) registered with all masters after 5.054e-05s
I20260812 08:02:50.986395 19875 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:50790
time=2026-08-12T08:02:51.840Z level=INFO source=main.go:1678 msg="updated GOGC" old=100 new=75
time=2026-08-12T08:02:51.842Z level=INFO source=main.go:744 msg="Leaving GOMAXPROCS=4: CPU quota undefined" component=automaxprocs
time=2026-08-12T08:02:51.843Z level=INFO source=memlimit.go:198 msg="GOMEMLIMIT is updated" component=automemlimit package=github.com/KimMachineGun/automemlimit/memlimit GOMEMLIMIT=14142552883 previous=9223372036854775807
time=2026-08-12T08:02:51.844Z level=INFO source=main.go:851 msg="Starting Prometheus Server" mode=server version="(version=3.11.2, branch=HEAD, revision=f0f0fdd679dcd6df320b0558b20919f7cd44c407)"
time=2026-08-12T08:02:51.844Z level=INFO source=main.go:856 msg="operational information" build_context="(go=go1.26.2, platform=linux/amd64, user=root@d2684485c347, date=20260413-11:53:40, tags=netgo,builtinassets)" host_details="(Linux 5.4.0-1157-gcp #166~18.04.1-Ubuntu SMP Wed Jan 21 01:57:46 UTC 2026 x86_64 dist-test-slave-8w9v (none))" fd_limits="(soft=1048576, hard=1048576)" vm_limits="(soft=unlimited, hard=unlimited)"
I20260812 08:02:51.988335 20033 heartbeater.cc:499] Master 127.19.96.126:43045 was elected leader, sending a full tablet report...
time=2026-08-12T08:02:52.109Z level=INFO source=web.go:710 msg="Start listening for connections" component=web address=127.0.0.1:0
time=2026-08-12T08:02:52.112Z level=INFO source=main.go:1410 msg="Starting TSDB ..."
time=2026-08-12T08:02:52.119Z level=INFO source=tls_config.go:354 msg="Listening on" component=web address=127.0.0.1:46433
time=2026-08-12T08:02:52.120Z level=INFO source=tls_config.go:357 msg="TLS is disabled." component=web http2=false address=127.0.0.1:46433
time=2026-08-12T08:02:52.128Z level=INFO source=head.go:698 msg="Replaying on-disk memory mappable chunks if any" component=tsdb
time=2026-08-12T08:02:52.130Z level=INFO source=head.go:784 msg="On-disk memory mappable chunks replay completed" component=tsdb duration=11.501µs
time=2026-08-12T08:02:52.130Z level=INFO source=head.go:792 msg="Replaying WAL, this may take a while" component=tsdb
time=2026-08-12T08:02:52.132Z level=INFO source=head.go:865 msg="WAL segment loaded" component=tsdb segment=0 maxSegment=0 duration=1.453336ms
time=2026-08-12T08:02:52.132Z level=INFO source=head.go:902 msg="WAL replay completed" component=tsdb checkpoint_replay_duration=149.049µs wal_replay_duration=1.55473ms wbl_replay_duration=352ns chunk_snapshot_load_duration=0s mmap_chunk_replay_duration=11.501µs total_replay_duration=1.809663ms
time=2026-08-12T08:02:52.137Z level=INFO source=main.go:1431 msg="filesystem information" fs_type=OVERLAYFS_SUPER_MAGIC
time=2026-08-12T08:02:52.137Z level=INFO source=main.go:1434 msg="TSDB started"
time=2026-08-12T08:02:52.137Z level=INFO source=main.go:1632 msg="Loading configuration file" filename=/tmp/dist-test-task3yToRy/test-tmp/prometheus_sd-itest.0.PrometheusSDITest.BasicSDAndScrape.1786521770778185-19841-0/prometheus.yml
time=2026-08-12T08:02:52.138Z level=INFO source=main.go:1048 msg="TSDB retention updated" duration=15d size=0B percentage=0
time=2026-08-12T08:02:52.143Z level=INFO source=main.go:1671 msg="Completed loading of configuration file" db_storage=54.163µs remote_storage=3.112µs web_handler=1.013µs query_engine=1.952µs scrape=4.680369ms scrape_sd=303.158µs notify=2.6µs notify_sd=1.374µs rules=3.041µs tracing=12.253µs filename=/tmp/dist-test-task3yToRy/test-tmp/prometheus_sd-itest.0.PrometheusSDITest.BasicSDAndScrape.1786521770778185-19841-0/prometheus.yml totalDuration=5.448648ms
time=2026-08-12T08:02:52.143Z level=INFO source=main.go:1395 msg="Server is ready to receive web requests."
time=2026-08-12T08:02:52.143Z level=INFO source=manager.go:209 msg="Starting rule manager..." component="rule manager"
time=2026-08-12T08:02:58.193Z level=WARN source=main.go:1191 msg="Received an OS signal, exiting gracefully..." signal=terminated
time=2026-08-12T08:02:58.193Z level=INFO source=main.go:1216 msg="Stopping scrape discovery manager..."
time=2026-08-12T08:02:58.193Z level=INFO source=main.go:1230 msg="Stopping notify discovery manager..."
time=2026-08-12T08:02:58.193Z level=INFO source=main.go:1245 msg="Stopping rule manager manager..."
time=2026-08-12T08:02:58.193Z level=INFO source=manager.go:223 msg="Stopping rule manager..." component="rule manager"
time=2026-08-12T08:02:58.193Z level=INFO source=manager.go:239 msg="Rule manager stopped" component="rule manager"
time=2026-08-12T08:02:58.193Z level=INFO source=main.go:1269 msg="Stopping scrape manager..."
time=2026-08-12T08:02:58.193Z level=INFO source=main.go:1241 msg="Rule manager stopped"
time=2026-08-12T08:02:58.194Z level=INFO source=main.go:1284 msg="Stopping tracing manager..."
time=2026-08-12T08:02:58.194Z level=INFO source=main.go:1369 msg="Stopping reloaders..."
time=2026-08-12T08:02:58.194Z level=INFO source=main.go:1458 msg="Stopping storage..."
time=2026-08-12T08:02:58.194Z level=INFO source=main.go:1261 msg="Scrape manager stopped"
time=2026-08-12T08:02:58.193Z level=INFO source=main.go:1226 msg="Notify discovery manager stopped"
time=2026-08-12T08:02:58.194Z level=INFO source=main.go:1361 msg="Reloaders stopped"
time=2026-08-12T08:02:58.194Z level=INFO source=main.go:1280 msg="Tracing manager stopped"
time=2026-08-12T08:02:58.194Z level=INFO source=main.go:1212 msg="Scrape discovery manager stopped"
time=2026-08-12T08:02:58.199Z level=INFO source=main.go:1536 msg="Stopping web handler..."
time=2026-08-12T08:02:58.199Z level=INFO source=main.go:1559 msg="Stopping notifier manager..."
time=2026-08-12T08:02:58.199Z level=INFO source=main.go:1454 msg="TSDB stopped"
time=2026-08-12T08:02:58.199Z level=INFO source=main.go:1532 msg="Web handler stopped"
time=2026-08-12T08:02:58.199Z level=INFO source=manager.go:321 msg="Stopping notification manager..." component=notifier
time=2026-08-12T08:02:58.201Z level=INFO source=main.go:1555 msg="Notifier manager stopped"
time=2026-08-12T08:02:58.201Z level=INFO source=main.go:1570 msg="See you next time!"
I20260812 08:02:58.214152 19841 tablet_server.cc:179] TabletServer@127.19.96.65:0 shutting down...
I20260812 08:02:58.220096 19841 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260812 08:02:58.233615 19841 tablet_server.cc:196] TabletServer@127.19.96.65:0 shutdown complete.
I20260812 08:02:58.237685 19841 master.cc:562] Master@127.19.96.126:43045 shutting down...
I20260812 08:02:58.242175 19841 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 92843689dd954d84a9f63fb483566581 [term 1 LEADER]: Raft consensus shutting down.
I20260812 08:02:58.242470 19841 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 92843689dd954d84a9f63fb483566581 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 08:02:58.242547 19841 tablet_replica.cc:333] T 00000000000000000000000000000000 P 92843689dd954d84a9f63fb483566581: stopping tablet replica
I20260812 08:02:58.255813 19841 master.cc:584] Master@127.19.96.126:43045 shutdown complete.
[       OK ] PrometheusSDITest.BasicSDAndScrape (7474 ms)
[ RUN      ] PrometheusSDITest.SDLabels
I20260812 08:02:58.261945 19841 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.19.96.126:41191
I20260812 08:02:58.262276 19841 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 08:02:58.264473 20062 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:02:58.264492 20060 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:02:58.264706 19841 server_base.cc:1061] running on GCE node
W20260812 08:02:58.264729 20059 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:02:58.264971 19841 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260812 08:02:58.265031 19841 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260812 08:02:58.265056 19841 hybrid_clock.cc:648] HybridClock initialized: now 1786521778265056 us; error 0 us; skew 500 ppm
I20260812 08:02:58.265998 19841 webserver.cc:533] Webserver started at http://127.19.96.126:42931/ using document root <none> and password file <none>
I20260812 08:02:58.266198 19841 fs_manager.cc:362] Metadata directory not provided
I20260812 08:02:58.266276 19841 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 08:02:58.266362 19841 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 08:02:58.266767 19841 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task3yToRy/test-tmp/prometheus_sd-itest.0.PrometheusSDITest.SDLabels.1786521770778185-19841-0/minicluster-data/master-0-root/instance:
uuid: "a048ef5fde2142bc8f44adf06a992b5a"
format_stamp: "Formatted at 2026-08-12 08:02:58 on dist-test-slave-8w9v"
I20260812 08:02:58.268437 19841 fs_manager.cc:696] Time spent creating directory manager: real 0.001s	user 0.002s	sys 0.001s
I20260812 08:02:58.269434 20070 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:02:58.269687 19841 fs_manager.cc:730] Time spent opening block manager: real 0.001s	user 0.001s	sys 0.000s
I20260812 08:02:58.269778 19841 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task3yToRy/test-tmp/prometheus_sd-itest.0.PrometheusSDITest.SDLabels.1786521770778185-19841-0/minicluster-data/master-0-root
uuid: "a048ef5fde2142bc8f44adf06a992b5a"
format_stamp: "Formatted at 2026-08-12 08:02:58 on dist-test-slave-8w9v"
I20260812 08:02:58.269891 19841 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task3yToRy/test-tmp/prometheus_sd-itest.0.PrometheusSDITest.SDLabels.1786521770778185-19841-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-task3yToRy/test-tmp/prometheus_sd-itest.0.PrometheusSDITest.SDLabels.1786521770778185-19841-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-task3yToRy/test-tmp/prometheus_sd-itest.0.PrometheusSDITest.SDLabels.1786521770778185-19841-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 08:02:58.279170 19841 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:02:58.279661 19841 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:02:58.284703 19841 rpc_server.cc:307] RPC server started. Bound to: 127.19.96.126:41191
I20260812 08:02:58.286772 20132 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.19.96.126:41191 every 8 connection(s)
I20260812 08:02:58.287317 20134 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 08:02:58.289253 20134 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a048ef5fde2142bc8f44adf06a992b5a: Bootstrap starting.
I20260812 08:02:58.290182 20134 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a048ef5fde2142bc8f44adf06a992b5a: Neither blocks nor log segments found. Creating new log.
I20260812 08:02:58.291340 20134 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a048ef5fde2142bc8f44adf06a992b5a: No bootstrap required, opened a new log
I20260812 08:02:58.291880 20134 raft_consensus.cc:359] T 00000000000000000000000000000000 P a048ef5fde2142bc8f44adf06a992b5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a048ef5fde2142bc8f44adf06a992b5a" member_type: VOTER }
I20260812 08:02:58.292001 20134 raft_consensus.cc:385] T 00000000000000000000000000000000 P a048ef5fde2142bc8f44adf06a992b5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 08:02:58.292053 20134 raft_consensus.cc:740] T 00000000000000000000000000000000 P a048ef5fde2142bc8f44adf06a992b5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a048ef5fde2142bc8f44adf06a992b5a, State: Initialized, Role: FOLLOWER
I20260812 08:02:58.292268 20134 consensus_queue.cc:260] T 00000000000000000000000000000000 P a048ef5fde2142bc8f44adf06a992b5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a048ef5fde2142bc8f44adf06a992b5a" member_type: VOTER }
I20260812 08:02:58.292363 20134 raft_consensus.cc:399] T 00000000000000000000000000000000 P a048ef5fde2142bc8f44adf06a992b5a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 08:02:58.292443 20134 raft_consensus.cc:493] T 00000000000000000000000000000000 P a048ef5fde2142bc8f44adf06a992b5a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 08:02:58.292505 20134 raft_consensus.cc:3060] T 00000000000000000000000000000000 P a048ef5fde2142bc8f44adf06a992b5a [term 0 FOLLOWER]: Advancing to term 1
I20260812 08:02:58.293246 20134 raft_consensus.cc:515] T 00000000000000000000000000000000 P a048ef5fde2142bc8f44adf06a992b5a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a048ef5fde2142bc8f44adf06a992b5a" member_type: VOTER }
I20260812 08:02:58.293416 20134 leader_election.cc:304] T 00000000000000000000000000000000 P a048ef5fde2142bc8f44adf06a992b5a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a048ef5fde2142bc8f44adf06a992b5a; no voters: 
I20260812 08:02:58.293627 20134 leader_election.cc:290] T 00000000000000000000000000000000 P a048ef5fde2142bc8f44adf06a992b5a [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260812 08:02:58.293788 20139 raft_consensus.cc:2804] T 00000000000000000000000000000000 P a048ef5fde2142bc8f44adf06a992b5a [term 1 FOLLOWER]: Leader election won for term 1
I20260812 08:02:58.294032 20139 raft_consensus.cc:697] T 00000000000000000000000000000000 P a048ef5fde2142bc8f44adf06a992b5a [term 1 LEADER]: Becoming Leader. State: Replica: a048ef5fde2142bc8f44adf06a992b5a, State: Running, Role: LEADER
I20260812 08:02:58.294132 20134 sys_catalog.cc:565] T 00000000000000000000000000000000 P a048ef5fde2142bc8f44adf06a992b5a [sys.catalog]: configured and running, proceeding with master startup.
I20260812 08:02:58.294204 20139 consensus_queue.cc:237] T 00000000000000000000000000000000 P a048ef5fde2142bc8f44adf06a992b5a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a048ef5fde2142bc8f44adf06a992b5a" member_type: VOTER }
I20260812 08:02:58.294747 20140 sys_catalog.cc:455] T 00000000000000000000000000000000 P a048ef5fde2142bc8f44adf06a992b5a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a048ef5fde2142bc8f44adf06a992b5a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a048ef5fde2142bc8f44adf06a992b5a" member_type: VOTER } }
I20260812 08:02:58.294776 20141 sys_catalog.cc:455] T 00000000000000000000000000000000 P a048ef5fde2142bc8f44adf06a992b5a [sys.catalog]: SysCatalogTable state changed. Reason: New leader a048ef5fde2142bc8f44adf06a992b5a. Latest consensus state: current_term: 1 leader_uuid: "a048ef5fde2142bc8f44adf06a992b5a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a048ef5fde2142bc8f44adf06a992b5a" member_type: VOTER } }
I20260812 08:02:58.294854 20140 sys_catalog.cc:458] T 00000000000000000000000000000000 P a048ef5fde2142bc8f44adf06a992b5a [sys.catalog]: This master's current role is: LEADER
I20260812 08:02:58.294866 20141 sys_catalog.cc:458] T 00000000000000000000000000000000 P a048ef5fde2142bc8f44adf06a992b5a [sys.catalog]: This master's current role is: LEADER
I20260812 08:02:58.295145 20147 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260812 08:02:58.296429 20147 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260812 08:02:58.296609 19841 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260812 08:02:58.298436 20147 catalog_manager.cc:1383] Generated new cluster ID: 08271d5ae097427f8137f91280271f39
I20260812 08:02:58.298511 20147 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260812 08:02:58.318565 20147 catalog_manager.cc:1406] Generated new certificate authority record
I20260812 08:02:58.319271 20147 catalog_manager.cc:1540] Loading token signing keys...
I20260812 08:02:58.326747 20147 catalog_manager.cc:6092] T 00000000000000000000000000000000 P a048ef5fde2142bc8f44adf06a992b5a: Generated new TSK 0
I20260812 08:02:58.326995 20147 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260812 08:02:58.328936 19841 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 08:02:58.331115 20159 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:02:58.331163 19841 server_base.cc:1061] running on GCE node
W20260812 08:02:58.331195 20163 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:02:58.331306 20160 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:02:58.331614 19841 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260812 08:02:58.331681 19841 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260812 08:02:58.331707 19841 hybrid_clock.cc:648] HybridClock initialized: now 1786521778331706 us; error 0 us; skew 500 ppm
I20260812 08:02:58.332623 19841 webserver.cc:533] Webserver started at http://127.19.96.65:43271/ using document root <none> and password file <none>
I20260812 08:02:58.332808 19841 fs_manager.cc:362] Metadata directory not provided
I20260812 08:02:58.332882 19841 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 08:02:58.332963 19841 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 08:02:58.333361 19841 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task3yToRy/test-tmp/prometheus_sd-itest.0.PrometheusSDITest.SDLabels.1786521770778185-19841-0/minicluster-data/ts-0-root/instance:
uuid: "fa05c2fce65e4a3fb79eece342e0cbea"
format_stamp: "Formatted at 2026-08-12 08:02:58 on dist-test-slave-8w9v"
I20260812 08:02:58.334939 19841 fs_manager.cc:696] Time spent creating directory manager: real 0.001s	user 0.002s	sys 0.000s
I20260812 08:02:58.335911 20169 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:02:58.336251 19841 fs_manager.cc:730] Time spent opening block manager: real 0.001s	user 0.001s	sys 0.000s
I20260812 08:02:58.336340 19841 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task3yToRy/test-tmp/prometheus_sd-itest.0.PrometheusSDITest.SDLabels.1786521770778185-19841-0/minicluster-data/ts-0-root
uuid: "fa05c2fce65e4a3fb79eece342e0cbea"
format_stamp: "Formatted at 2026-08-12 08:02:58 on dist-test-slave-8w9v"
I20260812 08:02:58.336457 19841 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task3yToRy/test-tmp/prometheus_sd-itest.0.PrometheusSDITest.SDLabels.1786521770778185-19841-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-task3yToRy/test-tmp/prometheus_sd-itest.0.PrometheusSDITest.SDLabels.1786521770778185-19841-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-task3yToRy/test-tmp/prometheus_sd-itest.0.PrometheusSDITest.SDLabels.1786521770778185-19841-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 08:02:58.347344 19841 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:02:58.347847 19841 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:02:58.348261 19841 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260812 08:02:58.348758 19841 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 08:02:58.348822 19841 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:02:58.348873 19841 ts_tablet_manager.cc:616] Registered 0 tablets
I20260812 08:02:58.348932 19841 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:02:58.353753 19841 rpc_server.cc:307] RPC server started. Bound to: 127.19.96.65:45059
I20260812 08:02:58.353792 20245 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.19.96.65:45059 every 8 connection(s)
I20260812 08:02:58.367316 20246 heartbeater.cc:344] Connected to a master server at 127.19.96.126:41191
I20260812 08:02:58.367458 20246 heartbeater.cc:461] Registering TS with master...
I20260812 08:02:58.367762 20246 heartbeater.cc:507] Master 127.19.96.126:41191 requested a full tablet report, sending...
I20260812 08:02:58.368613 20090 ts_manager.cc:194] Registered new tserver with Master: fa05c2fce65e4a3fb79eece342e0cbea (127.19.96.65:45059)
I20260812 08:02:58.368688 19841 internal_mini_cluster.cc:371] 1 TS(s) registered with all masters after 0.014497928s
I20260812 08:02:58.369340 19841 internal_mini_cluster.cc:371] 1 TS(s) registered with all masters after 4.2705e-05s
I20260812 08:02:58.369570 20090 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:36696
time=2026-08-12T08:02:58.452Z level=INFO source=main.go:1678 msg="updated GOGC" old=100 new=75
time=2026-08-12T08:02:58.453Z level=INFO source=main.go:744 msg="Leaving GOMAXPROCS=4: CPU quota undefined" component=automaxprocs
time=2026-08-12T08:02:58.453Z level=INFO source=memlimit.go:198 msg="GOMEMLIMIT is updated" component=automemlimit package=github.com/KimMachineGun/automemlimit/memlimit GOMEMLIMIT=14142552883 previous=9223372036854775807
time=2026-08-12T08:02:58.453Z level=INFO source=main.go:851 msg="Starting Prometheus Server" mode=server version="(version=3.11.2, branch=HEAD, revision=f0f0fdd679dcd6df320b0558b20919f7cd44c407)"
time=2026-08-12T08:02:58.453Z level=INFO source=main.go:856 msg="operational information" build_context="(go=go1.26.2, platform=linux/amd64, user=root@d2684485c347, date=20260413-11:53:40, tags=netgo,builtinassets)" host_details="(Linux 5.4.0-1157-gcp #166~18.04.1-Ubuntu SMP Wed Jan 21 01:57:46 UTC 2026 x86_64 dist-test-slave-8w9v (none))" fd_limits="(soft=1048576, hard=1048576)" vm_limits="(soft=unlimited, hard=unlimited)"
time=2026-08-12T08:02:58.712Z level=INFO source=web.go:710 msg="Start listening for connections" component=web address=127.0.0.1:0
time=2026-08-12T08:02:58.713Z level=INFO source=main.go:1410 msg="Starting TSDB ..."
time=2026-08-12T08:02:58.716Z level=INFO source=tls_config.go:354 msg="Listening on" component=web address=127.0.0.1:45365
time=2026-08-12T08:02:58.716Z level=INFO source=tls_config.go:357 msg="TLS is disabled." component=web http2=false address=127.0.0.1:45365
time=2026-08-12T08:02:58.722Z level=INFO source=head.go:698 msg="Replaying on-disk memory mappable chunks if any" component=tsdb
time=2026-08-12T08:02:58.723Z level=INFO source=head.go:784 msg="On-disk memory mappable chunks replay completed" component=tsdb duration=4.118µs
time=2026-08-12T08:02:58.723Z level=INFO source=head.go:792 msg="Replaying WAL, this may take a while" component=tsdb
time=2026-08-12T08:02:58.724Z level=INFO source=head.go:865 msg="WAL segment loaded" component=tsdb segment=0 maxSegment=0 duration=753.638µs
time=2026-08-12T08:02:58.724Z level=INFO source=head.go:902 msg="WAL replay completed" component=tsdb checkpoint_replay_duration=110.316µs wal_replay_duration=828.967µs wbl_replay_duration=187ns chunk_snapshot_load_duration=0s mmap_chunk_replay_duration=4.118µs total_replay_duration=1.106768ms
time=2026-08-12T08:02:58.730Z level=INFO source=main.go:1431 msg="filesystem information" fs_type=OVERLAYFS_SUPER_MAGIC
time=2026-08-12T08:02:58.730Z level=INFO source=main.go:1434 msg="TSDB started"
time=2026-08-12T08:02:58.730Z level=INFO source=main.go:1632 msg="Loading configuration file" filename=/tmp/dist-test-task3yToRy/test-tmp/prometheus_sd-itest.0.PrometheusSDITest.SDLabels.1786521770778185-19841-0/prometheus.yml
time=2026-08-12T08:02:58.731Z level=INFO source=main.go:1048 msg="TSDB retention updated" duration=15d size=0B percentage=0
time=2026-08-12T08:02:58.733Z level=INFO source=main.go:1671 msg="Completed loading of configuration file" db_storage=109.824µs remote_storage=4.75µs web_handler=1.203µs query_engine=2.826µs scrape=2.242491ms scrape_sd=160.376µs notify=2.429µs notify_sd=1.558µs rules=2.897µs tracing=11.364µs filename=/tmp/dist-test-task3yToRy/test-tmp/prometheus_sd-itest.0.PrometheusSDITest.SDLabels.1786521770778185-19841-0/prometheus.yml totalDuration=3.249752ms
time=2026-08-12T08:02:58.733Z level=INFO source=main.go:1395 msg="Server is ready to receive web requests."
time=2026-08-12T08:02:58.733Z level=INFO source=manager.go:209 msg="Starting rule manager..." component="rule manager"
I20260812 08:02:59.374661 20246 heartbeater.cc:499] Master 127.19.96.126:41191 was elected leader, sending a full tablet report...
time=2026-08-12T08:03:05.584Z level=WARN source=main.go:1191 msg="Received an OS signal, exiting gracefully..." signal=terminated
time=2026-08-12T08:03:05.585Z level=INFO source=main.go:1216 msg="Stopping scrape discovery manager..."
time=2026-08-12T08:03:05.585Z level=INFO source=main.go:1230 msg="Stopping notify discovery manager..."
time=2026-08-12T08:03:05.585Z level=INFO source=main.go:1245 msg="Stopping rule manager manager..."
time=2026-08-12T08:03:05.585Z level=INFO source=manager.go:223 msg="Stopping rule manager..." component="rule manager"
time=2026-08-12T08:03:05.585Z level=INFO source=manager.go:239 msg="Rule manager stopped" component="rule manager"
time=2026-08-12T08:03:05.585Z level=INFO source=main.go:1269 msg="Stopping scrape manager..."
time=2026-08-12T08:03:05.585Z level=INFO source=main.go:1212 msg="Scrape discovery manager stopped"
time=2026-08-12T08:03:05.585Z level=INFO source=main.go:1226 msg="Notify discovery manager stopped"
time=2026-08-12T08:03:05.585Z level=INFO source=main.go:1241 msg="Rule manager stopped"
time=2026-08-12T08:03:05.585Z level=INFO source=main.go:1284 msg="Stopping tracing manager..."
time=2026-08-12T08:03:05.585Z level=INFO source=main.go:1369 msg="Stopping reloaders..."
time=2026-08-12T08:03:05.585Z level=INFO source=main.go:1458 msg="Stopping storage..."
time=2026-08-12T08:03:05.586Z level=INFO source=main.go:1361 msg="Reloaders stopped"
time=2026-08-12T08:03:05.585Z level=INFO source=main.go:1261 msg="Scrape manager stopped"
time=2026-08-12T08:03:05.586Z level=INFO source=main.go:1280 msg="Tracing manager stopped"
time=2026-08-12T08:03:05.591Z level=INFO source=main.go:1536 msg="Stopping web handler..."
time=2026-08-12T08:03:05.591Z level=INFO source=main.go:1559 msg="Stopping notifier manager..."
time=2026-08-12T08:03:05.591Z level=INFO source=manager.go:321 msg="Stopping notification manager..." component=notifier
time=2026-08-12T08:03:05.591Z level=INFO source=main.go:1555 msg="Notifier manager stopped"
time=2026-08-12T08:03:05.591Z level=INFO source=main.go:1454 msg="TSDB stopped"
time=2026-08-12T08:03:05.591Z level=INFO source=main.go:1532 msg="Web handler stopped"
time=2026-08-12T08:03:05.591Z level=INFO source=main.go:1570 msg="See you next time!"
I20260812 08:03:05.605415 19841 tablet_server.cc:179] TabletServer@127.19.96.65:0 shutting down...
I20260812 08:03:05.609548 19841 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260812 08:03:05.622036 19841 tablet_server.cc:196] TabletServer@127.19.96.65:0 shutdown complete.
I20260812 08:03:05.624320 19841 master.cc:562] Master@127.19.96.126:41191 shutting down...
I20260812 08:03:05.627970 19841 raft_consensus.cc:2243] T 00000000000000000000000000000000 P a048ef5fde2142bc8f44adf06a992b5a [term 1 LEADER]: Raft consensus shutting down.
I20260812 08:03:05.628216 19841 raft_consensus.cc:2272] T 00000000000000000000000000000000 P a048ef5fde2142bc8f44adf06a992b5a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 08:03:05.628290 19841 tablet_replica.cc:333] T 00000000000000000000000000000000 P a048ef5fde2142bc8f44adf06a992b5a: stopping tablet replica
I20260812 08:03:05.641098 19841 master.cc:584] Master@127.19.96.126:41191 shutdown complete.
[       OK ] PrometheusSDITest.SDLabels (7384 ms)
[ RUN      ] PrometheusSDITest.TserverMetricContinuityDuringMasterRestart
I20260812 08:03:05.646703 19841 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.19.96.126:39063
I20260812 08:03:05.647076 19841 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 08:03:05.649304 20276 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:03:05.649363 20277 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:03:05.649494 20279 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:03:05.649634 19841 server_base.cc:1061] running on GCE node
I20260812 08:03:05.649792 19841 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260812 08:03:05.649829 19841 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260812 08:03:05.649847 19841 hybrid_clock.cc:648] HybridClock initialized: now 1786521785649847 us; error 0 us; skew 500 ppm
I20260812 08:03:05.650768 19841 webserver.cc:533] Webserver started at http://127.19.96.126:42289/ using document root <none> and password file <none>
I20260812 08:03:05.650940 19841 fs_manager.cc:362] Metadata directory not provided
I20260812 08:03:05.650992 19841 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 08:03:05.651055 19841 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 08:03:05.651482 19841 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task3yToRy/test-tmp/prometheus_sd-itest.0.PrometheusSDITest.TserverMetricContinuityDuringMasterRestart.1786521770778185-19841-0/minicluster-data/master-0-root/instance:
uuid: "b00b255bda3a4528a777880bf0d2cbab"
format_stamp: "Formatted at 2026-08-12 08:03:05 on dist-test-slave-8w9v"
I20260812 08:03:05.653244 19841 fs_manager.cc:696] Time spent creating directory manager: real 0.002s	user 0.000s	sys 0.002s
I20260812 08:03:05.654310 20285 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:03:05.654578 19841 fs_manager.cc:730] Time spent opening block manager: real 0.001s	user 0.000s	sys 0.001s
I20260812 08:03:05.654652 19841 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task3yToRy/test-tmp/prometheus_sd-itest.0.PrometheusSDITest.TserverMetricContinuityDuringMasterRestart.1786521770778185-19841-0/minicluster-data/master-0-root
uuid: "b00b255bda3a4528a777880bf0d2cbab"
format_stamp: "Formatted at 2026-08-12 08:03:05 on dist-test-slave-8w9v"
I20260812 08:03:05.654726 19841 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task3yToRy/test-tmp/prometheus_sd-itest.0.PrometheusSDITest.TserverMetricContinuityDuringMasterRestart.1786521770778185-19841-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-task3yToRy/test-tmp/prometheus_sd-itest.0.PrometheusSDITest.TserverMetricContinuityDuringMasterRestart.1786521770778185-19841-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-task3yToRy/test-tmp/prometheus_sd-itest.0.PrometheusSDITest.TserverMetricContinuityDuringMasterRestart.1786521770778185-19841-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 08:03:05.661795 19841 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:03:05.662233 19841 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:03:05.667498 19841 rpc_server.cc:307] RPC server started. Bound to: 127.19.96.126:39063
I20260812 08:03:05.668826 20348 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.19.96.126:39063 every 8 connection(s)
I20260812 08:03:05.669325 20349 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 08:03:05.671207 20349 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b00b255bda3a4528a777880bf0d2cbab: Bootstrap starting.
I20260812 08:03:05.672186 20349 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b00b255bda3a4528a777880bf0d2cbab: Neither blocks nor log segments found. Creating new log.
I20260812 08:03:05.673338 20349 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b00b255bda3a4528a777880bf0d2cbab: No bootstrap required, opened a new log
I20260812 08:03:05.673846 20349 raft_consensus.cc:359] T 00000000000000000000000000000000 P b00b255bda3a4528a777880bf0d2cbab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b00b255bda3a4528a777880bf0d2cbab" member_type: VOTER }
I20260812 08:03:05.673938 20349 raft_consensus.cc:385] T 00000000000000000000000000000000 P b00b255bda3a4528a777880bf0d2cbab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 08:03:05.673961 20349 raft_consensus.cc:740] T 00000000000000000000000000000000 P b00b255bda3a4528a777880bf0d2cbab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b00b255bda3a4528a777880bf0d2cbab, State: Initialized, Role: FOLLOWER
I20260812 08:03:05.674122 20349 consensus_queue.cc:260] T 00000000000000000000000000000000 P b00b255bda3a4528a777880bf0d2cbab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b00b255bda3a4528a777880bf0d2cbab" member_type: VOTER }
I20260812 08:03:05.674196 20349 raft_consensus.cc:399] T 00000000000000000000000000000000 P b00b255bda3a4528a777880bf0d2cbab [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 08:03:05.674240 20349 raft_consensus.cc:493] T 00000000000000000000000000000000 P b00b255bda3a4528a777880bf0d2cbab [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 08:03:05.674299 20349 raft_consensus.cc:3060] T 00000000000000000000000000000000 P b00b255bda3a4528a777880bf0d2cbab [term 0 FOLLOWER]: Advancing to term 1
I20260812 08:03:05.675073 20349 raft_consensus.cc:515] T 00000000000000000000000000000000 P b00b255bda3a4528a777880bf0d2cbab [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b00b255bda3a4528a777880bf0d2cbab" member_type: VOTER }
I20260812 08:03:05.675231 20349 leader_election.cc:304] T 00000000000000000000000000000000 P b00b255bda3a4528a777880bf0d2cbab [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b00b255bda3a4528a777880bf0d2cbab; no voters: 
I20260812 08:03:05.675455 20349 leader_election.cc:290] T 00000000000000000000000000000000 P b00b255bda3a4528a777880bf0d2cbab [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260812 08:03:05.675616 20352 raft_consensus.cc:2804] T 00000000000000000000000000000000 P b00b255bda3a4528a777880bf0d2cbab [term 1 FOLLOWER]: Leader election won for term 1
I20260812 08:03:05.675856 20352 raft_consensus.cc:697] T 00000000000000000000000000000000 P b00b255bda3a4528a777880bf0d2cbab [term 1 LEADER]: Becoming Leader. State: Replica: b00b255bda3a4528a777880bf0d2cbab, State: Running, Role: LEADER
I20260812 08:03:05.675942 20349 sys_catalog.cc:565] T 00000000000000000000000000000000 P b00b255bda3a4528a777880bf0d2cbab [sys.catalog]: configured and running, proceeding with master startup.
I20260812 08:03:05.676020 20352 consensus_queue.cc:237] T 00000000000000000000000000000000 P b00b255bda3a4528a777880bf0d2cbab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b00b255bda3a4528a777880bf0d2cbab" member_type: VOTER }
I20260812 08:03:05.676549 20356 sys_catalog.cc:455] T 00000000000000000000000000000000 P b00b255bda3a4528a777880bf0d2cbab [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b00b255bda3a4528a777880bf0d2cbab" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b00b255bda3a4528a777880bf0d2cbab" member_type: VOTER } }
I20260812 08:03:05.676609 20357 sys_catalog.cc:455] T 00000000000000000000000000000000 P b00b255bda3a4528a777880bf0d2cbab [sys.catalog]: SysCatalogTable state changed. Reason: New leader b00b255bda3a4528a777880bf0d2cbab. Latest consensus state: current_term: 1 leader_uuid: "b00b255bda3a4528a777880bf0d2cbab" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b00b255bda3a4528a777880bf0d2cbab" member_type: VOTER } }
I20260812 08:03:05.676745 20357 sys_catalog.cc:458] T 00000000000000000000000000000000 P b00b255bda3a4528a777880bf0d2cbab [sys.catalog]: This master's current role is: LEADER
I20260812 08:03:05.676713 20356 sys_catalog.cc:458] T 00000000000000000000000000000000 P b00b255bda3a4528a777880bf0d2cbab [sys.catalog]: This master's current role is: LEADER
I20260812 08:03:05.677237 20362 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260812 08:03:05.678223 20362 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260812 08:03:05.678567 19841 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260812 08:03:05.680783 20362 catalog_manager.cc:1383] Generated new cluster ID: 0868f51c32d5483192085998831e2fc3
I20260812 08:03:05.680845 20362 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260812 08:03:05.687537 20362 catalog_manager.cc:1406] Generated new certificate authority record
I20260812 08:03:05.688409 20362 catalog_manager.cc:1540] Loading token signing keys...
I20260812 08:03:05.695192 20362 catalog_manager.cc:6092] T 00000000000000000000000000000000 P b00b255bda3a4528a777880bf0d2cbab: Generated new TSK 0
I20260812 08:03:05.695484 20362 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260812 08:03:05.711167 19841 file_cache.cc:504] Constructed file cache file cache with capacity 419430
I20260812 08:03:05.713689 19841 server_base.cc:1061] running on GCE node
W20260812 08:03:05.713673 20378 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:03:05.713673 20380 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:03:05.713867 20377 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:03:05.714195 19841 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260812 08:03:05.714253 19841 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260812 08:03:05.714272 19841 hybrid_clock.cc:648] HybridClock initialized: now 1786521785714271 us; error 0 us; skew 500 ppm
I20260812 08:03:05.715186 19841 webserver.cc:533] Webserver started at http://127.19.96.65:46209/ using document root <none> and password file <none>
I20260812 08:03:05.715338 19841 fs_manager.cc:362] Metadata directory not provided
I20260812 08:03:05.715386 19841 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 08:03:05.715479 19841 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 08:03:05.715831 19841 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task3yToRy/test-tmp/prometheus_sd-itest.0.PrometheusSDITest.TserverMetricContinuityDuringMasterRestart.1786521770778185-19841-0/minicluster-data/ts-0-root/instance:
uuid: "c6f7877b7aab485e94108dcaf782a555"
format_stamp: "Formatted at 2026-08-12 08:03:05 on dist-test-slave-8w9v"
I20260812 08:03:05.717517 19841 fs_manager.cc:696] Time spent creating directory manager: real 0.001s	user 0.002s	sys 0.000s
I20260812 08:03:05.718541 20387 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:03:05.718868 19841 fs_manager.cc:730] Time spent opening block manager: real 0.001s	user 0.001s	sys 0.000s
I20260812 08:03:05.718957 19841 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task3yToRy/test-tmp/prometheus_sd-itest.0.PrometheusSDITest.TserverMetricContinuityDuringMasterRestart.1786521770778185-19841-0/minicluster-data/ts-0-root
uuid: "c6f7877b7aab485e94108dcaf782a555"
format_stamp: "Formatted at 2026-08-12 08:03:05 on dist-test-slave-8w9v"
I20260812 08:03:05.719063 19841 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task3yToRy/test-tmp/prometheus_sd-itest.0.PrometheusSDITest.TserverMetricContinuityDuringMasterRestart.1786521770778185-19841-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-task3yToRy/test-tmp/prometheus_sd-itest.0.PrometheusSDITest.TserverMetricContinuityDuringMasterRestart.1786521770778185-19841-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-task3yToRy/test-tmp/prometheus_sd-itest.0.PrometheusSDITest.TserverMetricContinuityDuringMasterRestart.1786521770778185-19841-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 08:03:05.727022 19841 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:03:05.727551 19841 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:03:05.727921 19841 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260812 08:03:05.728533 19841 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 08:03:05.728600 19841 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:03:05.728662 19841 ts_tablet_manager.cc:616] Registered 0 tablets
I20260812 08:03:05.728713 19841 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:03:05.733739 19841 rpc_server.cc:307] RPC server started. Bound to: 127.19.96.65:39629
I20260812 08:03:05.733790 20463 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.19.96.65:39629 every 8 connection(s)
I20260812 08:03:05.739625 20464 heartbeater.cc:344] Connected to a master server at 127.19.96.126:39063
I20260812 08:03:05.739760 20464 heartbeater.cc:461] Registering TS with master...
I20260812 08:03:05.740090 20464 heartbeater.cc:507] Master 127.19.96.126:39063 requested a full tablet report, sending...
I20260812 08:03:05.740914 20307 ts_manager.cc:194] Registered new tserver with Master: c6f7877b7aab485e94108dcaf782a555 (127.19.96.65:39629)
I20260812 08:03:05.741091 19841 internal_mini_cluster.cc:371] 1 TS(s) registered with all masters after 0.006879045s
I20260812 08:03:05.741715 19841 internal_mini_cluster.cc:371] 1 TS(s) registered with all masters after 4.2929e-05s
I20260812 08:03:05.741940 20307 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:46976
time=2026-08-12T08:03:05.827Z level=INFO source=main.go:1678 msg="updated GOGC" old=100 new=75
time=2026-08-12T08:03:05.829Z level=INFO source=main.go:744 msg="Leaving GOMAXPROCS=4: CPU quota undefined" component=automaxprocs
time=2026-08-12T08:03:05.830Z level=INFO source=memlimit.go:198 msg="GOMEMLIMIT is updated" component=automemlimit package=github.com/KimMachineGun/automemlimit/memlimit GOMEMLIMIT=14142552883 previous=9223372036854775807
time=2026-08-12T08:03:05.830Z level=INFO source=main.go:851 msg="Starting Prometheus Server" mode=server version="(version=3.11.2, branch=HEAD, revision=f0f0fdd679dcd6df320b0558b20919f7cd44c407)"
time=2026-08-12T08:03:05.830Z level=INFO source=main.go:856 msg="operational information" build_context="(go=go1.26.2, platform=linux/amd64, user=root@d2684485c347, date=20260413-11:53:40, tags=netgo,builtinassets)" host_details="(Linux 5.4.0-1157-gcp #166~18.04.1-Ubuntu SMP Wed Jan 21 01:57:46 UTC 2026 x86_64 dist-test-slave-8w9v (none))" fd_limits="(soft=1048576, hard=1048576)" vm_limits="(soft=unlimited, hard=unlimited)"
time=2026-08-12T08:03:06.046Z level=INFO source=web.go:710 msg="Start listening for connections" component=web address=127.0.0.1:0
time=2026-08-12T08:03:06.047Z level=INFO source=main.go:1410 msg="Starting TSDB ..."
time=2026-08-12T08:03:06.052Z level=INFO source=tls_config.go:354 msg="Listening on" component=web address=127.0.0.1:46269
time=2026-08-12T08:03:06.052Z level=INFO source=tls_config.go:357 msg="TLS is disabled." component=web http2=false address=127.0.0.1:46269
time=2026-08-12T08:03:06.053Z level=INFO source=head.go:698 msg="Replaying on-disk memory mappable chunks if any" component=tsdb
time=2026-08-12T08:03:06.054Z level=INFO source=head.go:784 msg="On-disk memory mappable chunks replay completed" component=tsdb duration=2.314µs
time=2026-08-12T08:03:06.054Z level=INFO source=head.go:792 msg="Replaying WAL, this may take a while" component=tsdb
time=2026-08-12T08:03:06.055Z level=INFO source=head.go:865 msg="WAL segment loaded" component=tsdb segment=0 maxSegment=0 duration=1.464116ms
time=2026-08-12T08:03:06.055Z level=INFO source=head.go:902 msg="WAL replay completed" component=tsdb checkpoint_replay_duration=64.677µs wal_replay_duration=1.590999ms wbl_replay_duration=201ns chunk_snapshot_load_duration=0s mmap_chunk_replay_duration=2.314µs total_replay_duration=1.709825ms
time=2026-08-12T08:03:06.063Z level=INFO source=main.go:1431 msg="filesystem information" fs_type=OVERLAYFS_SUPER_MAGIC
time=2026-08-12T08:03:06.063Z level=INFO source=main.go:1434 msg="TSDB started"
time=2026-08-12T08:03:06.063Z level=INFO source=main.go:1632 msg="Loading configuration file" filename=/tmp/dist-test-task3yToRy/test-tmp/prometheus_sd-itest.0.PrometheusSDITest.TserverMetricContinuityDuringMasterRestart.1786521770778185-19841-0/prometheus.yml
time=2026-08-12T08:03:06.063Z level=INFO source=main.go:1048 msg="TSDB retention updated" duration=15d size=0B percentage=0
time=2026-08-12T08:03:06.065Z level=INFO source=main.go:1671 msg="Completed loading of configuration file" db_storage=55.224µs remote_storage=2.629µs web_handler=863ns query_engine=1.567µs scrape=1.379067ms scrape_sd=137.7µs notify=3.434µs notify_sd=802ns rules=1.554µs tracing=7.774µs filename=/tmp/dist-test-task3yToRy/test-tmp/prometheus_sd-itest.0.PrometheusSDITest.TserverMetricContinuityDuringMasterRestart.1786521770778185-19841-0/prometheus.yml totalDuration=1.969684ms
time=2026-08-12T08:03:06.065Z level=INFO source=main.go:1395 msg="Server is ready to receive web requests."
time=2026-08-12T08:03:06.065Z level=INFO source=manager.go:209 msg="Starting rule manager..." component="rule manager"
I20260812 08:03:06.747136 20464 heartbeater.cc:499] Master 127.19.96.126:39063 was elected leader, sending a full tablet report...
I20260812 08:03:11.871198 19841 master.cc:562] Master@127.19.96.126:39063 shutting down...
I20260812 08:03:11.875006 19841 raft_consensus.cc:2243] T 00000000000000000000000000000000 P b00b255bda3a4528a777880bf0d2cbab [term 1 LEADER]: Raft consensus shutting down.
I20260812 08:03:11.875245 19841 raft_consensus.cc:2272] T 00000000000000000000000000000000 P b00b255bda3a4528a777880bf0d2cbab [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 08:03:11.875337 19841 tablet_replica.cc:333] T 00000000000000000000000000000000 P b00b255bda3a4528a777880bf0d2cbab: stopping tablet replica
I20260812 08:03:11.888463 19841 master.cc:584] Master@127.19.96.126:39063 shutdown complete.
I20260812 08:03:11.893424 19841 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 08:03:11.895864 20486 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:03:11.895818 20488 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:03:11.895921 19841 server_base.cc:1061] running on GCE node
W20260812 08:03:11.895829 20485 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:03:11.896423 19841 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260812 08:03:11.896477 19841 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260812 08:03:11.896524 19841 hybrid_clock.cc:648] HybridClock initialized: now 1786521791896523 us; error 0 us; skew 500 ppm
I20260812 08:03:11.897564 19841 webserver.cc:533] Webserver started at http://127.19.96.126:42289/ using document root <none> and password file <none>
I20260812 08:03:11.897778 19841 fs_manager.cc:362] Metadata directory not provided
I20260812 08:03:11.897821 19841 fs_manager.cc:365] Using existing metadata directory in first data directory
I20260812 08:03:11.899041 19841 fs_manager.cc:714] Time spent opening directory manager: real 0.001s	user 0.001s	sys 0.000s
I20260812 08:03:11.900063 20493 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:03:11.900348 19841 fs_manager.cc:730] Time spent opening block manager: real 0.001s	user 0.001s	sys 0.000s
I20260812 08:03:11.900426 19841 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task3yToRy/test-tmp/prometheus_sd-itest.0.PrometheusSDITest.TserverMetricContinuityDuringMasterRestart.1786521770778185-19841-0/minicluster-data/master-0-root
uuid: "b00b255bda3a4528a777880bf0d2cbab"
format_stamp: "Formatted at 2026-08-12 08:03:05 on dist-test-slave-8w9v"
I20260812 08:03:11.900497 19841 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task3yToRy/test-tmp/prometheus_sd-itest.0.PrometheusSDITest.TserverMetricContinuityDuringMasterRestart.1786521770778185-19841-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-task3yToRy/test-tmp/prometheus_sd-itest.0.PrometheusSDITest.TserverMetricContinuityDuringMasterRestart.1786521770778185-19841-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-task3yToRy/test-tmp/prometheus_sd-itest.0.PrometheusSDITest.TserverMetricContinuityDuringMasterRestart.1786521770778185-19841-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 08:03:11.917292 19841 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:03:11.917704 19841 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:03:11.922726 19841 rpc_server.cc:307] RPC server started. Bound to: 127.19.96.126:39063
I20260812 08:03:11.923504 20563 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.19.96.126:39063 every 8 connection(s)
I20260812 08:03:11.925534 20564 sys_catalog.cc:263] Verifying existing consensus state
I20260812 08:03:11.926275 20564 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b00b255bda3a4528a777880bf0d2cbab: Bootstrap starting.
I20260812 08:03:11.932605 20564 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b00b255bda3a4528a777880bf0d2cbab: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260812 08:03:11.933019 20564 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b00b255bda3a4528a777880bf0d2cbab: Bootstrap complete.
I20260812 08:03:11.933524 20564 raft_consensus.cc:359] T 00000000000000000000000000000000 P b00b255bda3a4528a777880bf0d2cbab [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b00b255bda3a4528a777880bf0d2cbab" member_type: VOTER }
I20260812 08:03:11.933629 20564 raft_consensus.cc:740] T 00000000000000000000000000000000 P b00b255bda3a4528a777880bf0d2cbab [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: b00b255bda3a4528a777880bf0d2cbab, State: Initialized, Role: FOLLOWER
I20260812 08:03:11.933806 20564 consensus_queue.cc:260] T 00000000000000000000000000000000 P b00b255bda3a4528a777880bf0d2cbab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b00b255bda3a4528a777880bf0d2cbab" member_type: VOTER }
I20260812 08:03:11.933900 20564 raft_consensus.cc:399] T 00000000000000000000000000000000 P b00b255bda3a4528a777880bf0d2cbab [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 08:03:11.933926 20564 raft_consensus.cc:493] T 00000000000000000000000000000000 P b00b255bda3a4528a777880bf0d2cbab [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 08:03:11.933957 20564 raft_consensus.cc:3060] T 00000000000000000000000000000000 P b00b255bda3a4528a777880bf0d2cbab [term 1 FOLLOWER]: Advancing to term 2
I20260812 08:03:11.934689 20564 raft_consensus.cc:515] T 00000000000000000000000000000000 P b00b255bda3a4528a777880bf0d2cbab [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b00b255bda3a4528a777880bf0d2cbab" member_type: VOTER }
I20260812 08:03:11.934803 20564 leader_election.cc:304] T 00000000000000000000000000000000 P b00b255bda3a4528a777880bf0d2cbab [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b00b255bda3a4528a777880bf0d2cbab; no voters: 
I20260812 08:03:11.934963 20564 leader_election.cc:290] T 00000000000000000000000000000000 P b00b255bda3a4528a777880bf0d2cbab [CANDIDATE]: Term 2 election: Requested vote from peers 
I20260812 08:03:11.935123 20567 raft_consensus.cc:2804] T 00000000000000000000000000000000 P b00b255bda3a4528a777880bf0d2cbab [term 2 FOLLOWER]: Leader election won for term 2
I20260812 08:03:11.935362 20567 raft_consensus.cc:697] T 00000000000000000000000000000000 P b00b255bda3a4528a777880bf0d2cbab [term 2 LEADER]: Becoming Leader. State: Replica: b00b255bda3a4528a777880bf0d2cbab, State: Running, Role: LEADER
I20260812 08:03:11.935510 20564 sys_catalog.cc:565] T 00000000000000000000000000000000 P b00b255bda3a4528a777880bf0d2cbab [sys.catalog]: configured and running, proceeding with master startup.
I20260812 08:03:11.935535 20567 consensus_queue.cc:237] T 00000000000000000000000000000000 P b00b255bda3a4528a777880bf0d2cbab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b00b255bda3a4528a777880bf0d2cbab" member_type: VOTER }
I20260812 08:03:11.936023 20568 sys_catalog.cc:455] T 00000000000000000000000000000000 P b00b255bda3a4528a777880bf0d2cbab [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "b00b255bda3a4528a777880bf0d2cbab" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b00b255bda3a4528a777880bf0d2cbab" member_type: VOTER } }
I20260812 08:03:11.936076 20570 sys_catalog.cc:455] T 00000000000000000000000000000000 P b00b255bda3a4528a777880bf0d2cbab [sys.catalog]: SysCatalogTable state changed. Reason: New leader b00b255bda3a4528a777880bf0d2cbab. Latest consensus state: current_term: 2 leader_uuid: "b00b255bda3a4528a777880bf0d2cbab" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b00b255bda3a4528a777880bf0d2cbab" member_type: VOTER } }
I20260812 08:03:11.936193 20568 sys_catalog.cc:458] T 00000000000000000000000000000000 P b00b255bda3a4528a777880bf0d2cbab [sys.catalog]: This master's current role is: LEADER
I20260812 08:03:11.936537 20570 sys_catalog.cc:458] T 00000000000000000000000000000000 P b00b255bda3a4528a777880bf0d2cbab [sys.catalog]: This master's current role is: LEADER
I20260812 08:03:11.936802 20578 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260812 08:03:11.937978 20578 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260812 08:03:11.938639 20578 catalog_manager.cc:1295] Loaded cluster ID: 0868f51c32d5483192085998831e2fc3
I20260812 08:03:11.938727 20578 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260812 08:03:11.940601 20578 catalog_manager.cc:1540] Loading token signing keys...
time=2026-08-12T08:03:11.940Z level=WARN source=main.go:1191 msg="Received an OS signal, exiting gracefully..." signal=terminated
time=2026-08-12T08:03:11.941Z level=INFO source=main.go:1216 msg="Stopping scrape discovery manager..."
time=2026-08-12T08:03:11.941Z level=INFO source=main.go:1230 msg="Stopping notify discovery manager..."
time=2026-08-12T08:03:11.941Z level=INFO source=main.go:1245 msg="Stopping rule manager manager..."
time=2026-08-12T08:03:11.941Z level=INFO source=manager.go:223 msg="Stopping rule manager..." component="rule manager"
time=2026-08-12T08:03:11.941Z level=INFO source=main.go:1212 msg="Scrape discovery manager stopped"
time=2026-08-12T08:03:11.941Z level=INFO source=manager.go:239 msg="Rule manager stopped" component="rule manager"
time=2026-08-12T08:03:11.941Z level=INFO source=main.go:1269 msg="Stopping scrape manager..."
time=2026-08-12T08:03:11.941Z level=INFO source=main.go:1241 msg="Rule manager stopped"
I20260812 08:03:11.941286 20578 catalog_manager.cc:6103] T 00000000000000000000000000000000 P b00b255bda3a4528a777880bf0d2cbab: Loaded TSK: 0
I20260812 08:03:11.941946 20578 catalog_manager.cc:1550] Initializing in-progress tserver states...
time=2026-08-12T08:03:11.942Z level=INFO source=main.go:1226 msg="Notify discovery manager stopped"
time=2026-08-12T08:03:11.942Z level=INFO source=main.go:1261 msg="Scrape manager stopped"
time=2026-08-12T08:03:11.942Z level=INFO source=main.go:1284 msg="Stopping tracing manager..."
time=2026-08-12T08:03:11.942Z level=INFO source=main.go:1369 msg="Stopping reloaders..."
time=2026-08-12T08:03:11.942Z level=INFO source=main.go:1458 msg="Stopping storage..."
time=2026-08-12T08:03:11.942Z level=INFO source=main.go:1361 msg="Reloaders stopped"
time=2026-08-12T08:03:11.942Z level=INFO source=main.go:1280 msg="Tracing manager stopped"
time=2026-08-12T08:03:11.947Z level=INFO source=main.go:1536 msg="Stopping web handler..."
time=2026-08-12T08:03:11.947Z level=INFO source=main.go:1559 msg="Stopping notifier manager..."
time=2026-08-12T08:03:11.947Z level=INFO source=manager.go:321 msg="Stopping notification manager..." component=notifier
time=2026-08-12T08:03:11.947Z level=INFO source=main.go:1454 msg="TSDB stopped"
time=2026-08-12T08:03:11.947Z level=INFO source=main.go:1555 msg="Notifier manager stopped"
time=2026-08-12T08:03:11.948Z level=INFO source=main.go:1532 msg="Web handler stopped"
time=2026-08-12T08:03:11.948Z level=INFO source=main.go:1570 msg="See you next time!"
I20260812 08:03:11.961539 19841 tablet_server.cc:179] TabletServer@127.19.96.65:0 shutting down...
I20260812 08:03:11.965588 19841 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260812 08:03:11.977705 19841 tablet_server.cc:196] TabletServer@127.19.96.65:0 shutdown complete.
I20260812 08:03:11.979938 19841 master.cc:562] Master@127.19.96.126:39063 shutting down...
I20260812 08:03:11.984094 19841 raft_consensus.cc:2243] T 00000000000000000000000000000000 P b00b255bda3a4528a777880bf0d2cbab [term 2 LEADER]: Raft consensus shutting down.
I20260812 08:03:11.984364 19841 raft_consensus.cc:2272] T 00000000000000000000000000000000 P b00b255bda3a4528a777880bf0d2cbab [term 2 FOLLOWER]: Raft consensus is shut down!
I20260812 08:03:11.984496 19841 tablet_replica.cc:333] stopping tablet replica
I20260812 08:03:11.997401 19841 master.cc:584] Master@127.19.96.126:39063 shutdown complete.
[       OK ] PrometheusSDITest.TserverMetricContinuityDuringMasterRestart (6355 ms)
[----------] 3 tests from PrometheusSDITest (21215 ms total)

[----------] 2 tests from MultiMasterPrometheusSDITest
[ RUN      ] MultiMasterPrometheusSDITest.LeaderFollowerSDBehavior
I20260812 08:03:12.002650 19841 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.19.96.126:36207,127.19.96.125:40969,127.19.96.124:37201
I20260812 08:03:12.003152 19841 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 08:03:12.004850 20586 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:03:12.004973 20591 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:03:12.004980 20587 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:03:12.005610 19841 server_base.cc:1061] running on GCE node
I20260812 08:03:12.005784 19841 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260812 08:03:12.005822 19841 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260812 08:03:12.005838 19841 hybrid_clock.cc:648] HybridClock initialized: now 1786521792005838 us; error 0 us; skew 500 ppm
I20260812 08:03:12.006721 19841 webserver.cc:533] Webserver started at http://127.19.96.126:46565/ using document root <none> and password file <none>
I20260812 08:03:12.006877 19841 fs_manager.cc:362] Metadata directory not provided
I20260812 08:03:12.006919 19841 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 08:03:12.006974 19841 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 08:03:12.007305 19841 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task3yToRy/test-tmp/prometheus_sd-itest.0.MultiMasterPrometheusSDITest.LeaderFollowerSDBehavior.1786521770778185-19841-0/minicluster-data/master-0-root/instance:
uuid: "9be450af28cb4085992ad5e0ebb480b9"
format_stamp: "Formatted at 2026-08-12 08:03:12 on dist-test-slave-8w9v"
I20260812 08:03:12.009198 19841 fs_manager.cc:696] Time spent creating directory manager: real 0.002s	user 0.000s	sys 0.002s
I20260812 08:03:12.010181 20596 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:03:12.010517 19841 fs_manager.cc:730] Time spent opening block manager: real 0.001s	user 0.000s	sys 0.001s
I20260812 08:03:12.010586 19841 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task3yToRy/test-tmp/prometheus_sd-itest.0.MultiMasterPrometheusSDITest.LeaderFollowerSDBehavior.1786521770778185-19841-0/minicluster-data/master-0-root
uuid: "9be450af28cb4085992ad5e0ebb480b9"
format_stamp: "Formatted at 2026-08-12 08:03:12 on dist-test-slave-8w9v"
I20260812 08:03:12.010649 19841 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task3yToRy/test-tmp/prometheus_sd-itest.0.MultiMasterPrometheusSDITest.LeaderFollowerSDBehavior.1786521770778185-19841-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-task3yToRy/test-tmp/prometheus_sd-itest.0.MultiMasterPrometheusSDITest.LeaderFollowerSDBehavior.1786521770778185-19841-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-task3yToRy/test-tmp/prometheus_sd-itest.0.MultiMasterPrometheusSDITest.LeaderFollowerSDBehavior.1786521770778185-19841-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 08:03:12.019112 19841 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:03:12.019589 19841 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:03:12.025072 19841 rpc_server.cc:307] RPC server started. Bound to: 127.19.96.126:36207
I20260812 08:03:12.025733 20657 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.19.96.126:36207 every 8 connection(s)
I20260812 08:03:12.026378 19841 file_cache.cc:504] Constructed file cache file cache with capacity 419430
I20260812 08:03:12.026525 20659 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 08:03:12.028122 20659 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.96.126" port: 36207 } has no permanent_uuid. Determining permanent_uuid...
W20260812 08:03:12.028398 20661 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:03:12.028424 20662 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:03:12.028829 20664 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:03:12.029112 19841 server_base.cc:1061] running on GCE node
I20260812 08:03:12.029376 19841 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260812 08:03:12.029438 19841 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260812 08:03:12.029464 19841 hybrid_clock.cc:648] HybridClock initialized: now 1786521792029464 us; error 0 us; skew 500 ppm
I20260812 08:03:12.030552 19841 webserver.cc:533] Webserver started at http://127.19.96.125:43189/ using document root <none> and password file <none>
I20260812 08:03:12.030746 19841 fs_manager.cc:362] Metadata directory not provided
I20260812 08:03:12.030827 19841 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 08:03:12.030917 19841 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 08:03:12.031415 19841 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task3yToRy/test-tmp/prometheus_sd-itest.0.MultiMasterPrometheusSDITest.LeaderFollowerSDBehavior.1786521770778185-19841-0/minicluster-data/master-1-root/instance:
uuid: "8176988cf19f4bb199c5891c11aade49"
format_stamp: "Formatted at 2026-08-12 08:03:12 on dist-test-slave-8w9v"
I20260812 08:03:12.033491 19841 fs_manager.cc:696] Time spent creating directory manager: real 0.002s	user 0.003s	sys 0.000s
I20260812 08:03:12.034610 20672 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:03:12.034874 19841 fs_manager.cc:730] Time spent opening block manager: real 0.001s	user 0.001s	sys 0.000s
I20260812 08:03:12.034971 19841 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task3yToRy/test-tmp/prometheus_sd-itest.0.MultiMasterPrometheusSDITest.LeaderFollowerSDBehavior.1786521770778185-19841-0/minicluster-data/master-1-root
uuid: "8176988cf19f4bb199c5891c11aade49"
format_stamp: "Formatted at 2026-08-12 08:03:12 on dist-test-slave-8w9v"
I20260812 08:03:12.035069 19841 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task3yToRy/test-tmp/prometheus_sd-itest.0.MultiMasterPrometheusSDITest.LeaderFollowerSDBehavior.1786521770778185-19841-0/minicluster-data/master-1-root
metadata directory: /tmp/dist-test-task3yToRy/test-tmp/prometheus_sd-itest.0.MultiMasterPrometheusSDITest.LeaderFollowerSDBehavior.1786521770778185-19841-0/minicluster-data/master-1-root
1 data directories: /tmp/dist-test-task3yToRy/test-tmp/prometheus_sd-itest.0.MultiMasterPrometheusSDITest.LeaderFollowerSDBehavior.1786521770778185-19841-0/minicluster-data/master-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 08:03:12.042162 20659 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.96.125" port: 40969 } has no permanent_uuid. Determining permanent_uuid...
W20260812 08:03:12.043089 20598 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.19.96.125:40969: connect: Connection refused (error 111)
W20260812 08:03:12.044025 20659 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.19.96.125:40969: Network error: Client connection negotiation failed: client connection to 127.19.96.125:40969: connect: Connection refused (error 111)
I20260812 08:03:12.049623 19841 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:03:12.050168 19841 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:03:12.055071 19841 rpc_server.cc:307] RPC server started. Bound to: 127.19.96.125:40969
I20260812 08:03:12.055140 20746 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.19.96.125:40969 every 8 connection(s)
I20260812 08:03:12.055905 19841 file_cache.cc:504] Constructed file cache file cache with capacity 419430
I20260812 08:03:12.056079 20747 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 08:03:12.057582 20747 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.96.126" port: 36207 } has no permanent_uuid. Determining permanent_uuid...
W20260812 08:03:12.057803 20750 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:03:12.057830 20751 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:03:12.057855 20753 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:03:12.058377 19841 server_base.cc:1061] running on GCE node
I20260812 08:03:12.058593 19841 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260812 08:03:12.058641 19841 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260812 08:03:12.058657 19841 hybrid_clock.cc:648] HybridClock initialized: now 1786521792058657 us; error 0 us; skew 500 ppm
I20260812 08:03:12.059561 19841 webserver.cc:533] Webserver started at http://127.19.96.124:46483/ using document root <none> and password file <none>
I20260812 08:03:12.059720 19841 fs_manager.cc:362] Metadata directory not provided
I20260812 08:03:12.059780 19841 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 08:03:12.059856 19841 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 08:03:12.060279 19841 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task3yToRy/test-tmp/prometheus_sd-itest.0.MultiMasterPrometheusSDITest.LeaderFollowerSDBehavior.1786521770778185-19841-0/minicluster-data/master-2-root/instance:
uuid: "53e9fb42a73442ffa9babcbcdf8587f9"
format_stamp: "Formatted at 2026-08-12 08:03:12 on dist-test-slave-8w9v"
I20260812 08:03:12.062256 19841 fs_manager.cc:696] Time spent creating directory manager: real 0.002s	user 0.000s	sys 0.002s
I20260812 08:03:12.063030 20747 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.96.125" port: 40969 } has no permanent_uuid. Determining permanent_uuid...
I20260812 08:03:12.063728 20759 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:03:12.064049 19841 fs_manager.cc:730] Time spent opening block manager: real 0.001s	user 0.000s	sys 0.001s
I20260812 08:03:12.064174 19841 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task3yToRy/test-tmp/prometheus_sd-itest.0.MultiMasterPrometheusSDITest.LeaderFollowerSDBehavior.1786521770778185-19841-0/minicluster-data/master-2-root
uuid: "53e9fb42a73442ffa9babcbcdf8587f9"
format_stamp: "Formatted at 2026-08-12 08:03:12 on dist-test-slave-8w9v"
I20260812 08:03:12.064254 19841 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task3yToRy/test-tmp/prometheus_sd-itest.0.MultiMasterPrometheusSDITest.LeaderFollowerSDBehavior.1786521770778185-19841-0/minicluster-data/master-2-root
metadata directory: /tmp/dist-test-task3yToRy/test-tmp/prometheus_sd-itest.0.MultiMasterPrometheusSDITest.LeaderFollowerSDBehavior.1786521770778185-19841-0/minicluster-data/master-2-root
1 data directories: /tmp/dist-test-task3yToRy/test-tmp/prometheus_sd-itest.0.MultiMasterPrometheusSDITest.LeaderFollowerSDBehavior.1786521770778185-19841-0/minicluster-data/master-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 08:03:12.068022 20747 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.96.124" port: 37201 } has no permanent_uuid. Determining permanent_uuid...
W20260812 08:03:12.069460 20747 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.19.96.124:37201: Network error: Client connection negotiation failed: client connection to 127.19.96.124:37201: connect: Connection refused (error 111)
I20260812 08:03:12.071933 19841 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:03:12.072435 19841 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:03:12.077466 19841 rpc_server.cc:307] RPC server started. Bound to: 127.19.96.124:37201
I20260812 08:03:12.077527 20823 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.19.96.124:37201 every 8 connection(s)
I20260812 08:03:12.078004 19841 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260812 08:03:12.078482 20824 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 08:03:12.079602 20824 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.96.126" port: 36207 } has no permanent_uuid. Determining permanent_uuid...
I20260812 08:03:12.084281 20824 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.96.125" port: 40969 } has no permanent_uuid. Determining permanent_uuid...
I20260812 08:03:12.088207 20824 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.96.124" port: 37201 } has no permanent_uuid. Determining permanent_uuid...
I20260812 08:03:12.093376 20824 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 53e9fb42a73442ffa9babcbcdf8587f9: Bootstrap starting.
I20260812 08:03:12.094417 20824 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 53e9fb42a73442ffa9babcbcdf8587f9: Neither blocks nor log segments found. Creating new log.
I20260812 08:03:12.095636 20824 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 53e9fb42a73442ffa9babcbcdf8587f9: No bootstrap required, opened a new log
I20260812 08:03:12.096190 20824 raft_consensus.cc:359] T 00000000000000000000000000000000 P 53e9fb42a73442ffa9babcbcdf8587f9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9be450af28cb4085992ad5e0ebb480b9" member_type: VOTER last_known_addr { host: "127.19.96.126" port: 36207 } } peers { permanent_uuid: "8176988cf19f4bb199c5891c11aade49" member_type: VOTER last_known_addr { host: "127.19.96.125" port: 40969 } } peers { permanent_uuid: "53e9fb42a73442ffa9babcbcdf8587f9" member_type: VOTER last_known_addr { host: "127.19.96.124" port: 37201 } }
I20260812 08:03:12.096295 20824 raft_consensus.cc:385] T 00000000000000000000000000000000 P 53e9fb42a73442ffa9babcbcdf8587f9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 08:03:12.096318 20824 raft_consensus.cc:740] T 00000000000000000000000000000000 P 53e9fb42a73442ffa9babcbcdf8587f9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 53e9fb42a73442ffa9babcbcdf8587f9, State: Initialized, Role: FOLLOWER
I20260812 08:03:12.096540 20824 consensus_queue.cc:260] T 00000000000000000000000000000000 P 53e9fb42a73442ffa9babcbcdf8587f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9be450af28cb4085992ad5e0ebb480b9" member_type: VOTER last_known_addr { host: "127.19.96.126" port: 36207 } } peers { permanent_uuid: "8176988cf19f4bb199c5891c11aade49" member_type: VOTER last_known_addr { host: "127.19.96.125" port: 40969 } } peers { permanent_uuid: "53e9fb42a73442ffa9babcbcdf8587f9" member_type: VOTER last_known_addr { host: "127.19.96.124" port: 37201 } }
I20260812 08:03:12.096921 20831 sys_catalog.cc:455] T 00000000000000000000000000000000 P 53e9fb42a73442ffa9babcbcdf8587f9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9be450af28cb4085992ad5e0ebb480b9" member_type: VOTER last_known_addr { host: "127.19.96.126" port: 36207 } } peers { permanent_uuid: "8176988cf19f4bb199c5891c11aade49" member_type: VOTER last_known_addr { host: "127.19.96.125" port: 40969 } } peers { permanent_uuid: "53e9fb42a73442ffa9babcbcdf8587f9" member_type: VOTER last_known_addr { host: "127.19.96.124" port: 37201 } } }
I20260812 08:03:12.097043 20831 sys_catalog.cc:458] T 00000000000000000000000000000000 P 53e9fb42a73442ffa9babcbcdf8587f9 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 08:03:12.097033 20824 sys_catalog.cc:565] T 00000000000000000000000000000000 P 53e9fb42a73442ffa9babcbcdf8587f9 [sys.catalog]: configured and running, proceeding with master startup.
W20260812 08:03:12.099339 20843 catalog_manager.cc:1594] T 00000000000000000000000000000000 P 53e9fb42a73442ffa9babcbcdf8587f9: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 08:03:12.099439 20843 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 08:03:12.103256 20659 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.96.125" port: 40969 } attempt: 1
I20260812 08:03:12.108095 20659 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.96.124" port: 37201 } has no permanent_uuid. Determining permanent_uuid...
I20260812 08:03:12.113713 20659 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9be450af28cb4085992ad5e0ebb480b9: Bootstrap starting.
I20260812 08:03:12.114928 20659 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9be450af28cb4085992ad5e0ebb480b9: Neither blocks nor log segments found. Creating new log.
I20260812 08:03:12.116344 20659 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9be450af28cb4085992ad5e0ebb480b9: No bootstrap required, opened a new log
I20260812 08:03:12.116851 20659 raft_consensus.cc:359] T 00000000000000000000000000000000 P 9be450af28cb4085992ad5e0ebb480b9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9be450af28cb4085992ad5e0ebb480b9" member_type: VOTER last_known_addr { host: "127.19.96.126" port: 36207 } } peers { permanent_uuid: "8176988cf19f4bb199c5891c11aade49" member_type: VOTER last_known_addr { host: "127.19.96.125" port: 40969 } } peers { permanent_uuid: "53e9fb42a73442ffa9babcbcdf8587f9" member_type: VOTER last_known_addr { host: "127.19.96.124" port: 37201 } }
I20260812 08:03:12.116954 20659 raft_consensus.cc:385] T 00000000000000000000000000000000 P 9be450af28cb4085992ad5e0ebb480b9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 08:03:12.116977 20659 raft_consensus.cc:740] T 00000000000000000000000000000000 P 9be450af28cb4085992ad5e0ebb480b9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9be450af28cb4085992ad5e0ebb480b9, State: Initialized, Role: FOLLOWER
I20260812 08:03:12.117182 20659 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9be450af28cb4085992ad5e0ebb480b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9be450af28cb4085992ad5e0ebb480b9" member_type: VOTER last_known_addr { host: "127.19.96.126" port: 36207 } } peers { permanent_uuid: "8176988cf19f4bb199c5891c11aade49" member_type: VOTER last_known_addr { host: "127.19.96.125" port: 40969 } } peers { permanent_uuid: "53e9fb42a73442ffa9babcbcdf8587f9" member_type: VOTER last_known_addr { host: "127.19.96.124" port: 37201 } }
I20260812 08:03:12.117595 20846 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9be450af28cb4085992ad5e0ebb480b9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9be450af28cb4085992ad5e0ebb480b9" member_type: VOTER last_known_addr { host: "127.19.96.126" port: 36207 } } peers { permanent_uuid: "8176988cf19f4bb199c5891c11aade49" member_type: VOTER last_known_addr { host: "127.19.96.125" port: 40969 } } peers { permanent_uuid: "53e9fb42a73442ffa9babcbcdf8587f9" member_type: VOTER last_known_addr { host: "127.19.96.124" port: 37201 } } }
I20260812 08:03:12.117693 20846 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9be450af28cb4085992ad5e0ebb480b9 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 08:03:12.117738 20659 sys_catalog.cc:565] T 00000000000000000000000000000000 P 9be450af28cb4085992ad5e0ebb480b9 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 08:03:12.119516 19841 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 1
W20260812 08:03:12.120105 20859 catalog_manager.cc:1594] T 00000000000000000000000000000000 P 9be450af28cb4085992ad5e0ebb480b9: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 08:03:12.120261 20859 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 08:03:12.120627 20747 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.96.124" port: 37201 } attempt: 1
I20260812 08:03:12.126155 20747 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8176988cf19f4bb199c5891c11aade49: Bootstrap starting.
I20260812 08:03:12.127233 20747 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8176988cf19f4bb199c5891c11aade49: Neither blocks nor log segments found. Creating new log.
I20260812 08:03:12.128554 20747 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8176988cf19f4bb199c5891c11aade49: No bootstrap required, opened a new log
I20260812 08:03:12.129055 20747 raft_consensus.cc:359] T 00000000000000000000000000000000 P 8176988cf19f4bb199c5891c11aade49 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9be450af28cb4085992ad5e0ebb480b9" member_type: VOTER last_known_addr { host: "127.19.96.126" port: 36207 } } peers { permanent_uuid: "8176988cf19f4bb199c5891c11aade49" member_type: VOTER last_known_addr { host: "127.19.96.125" port: 40969 } } peers { permanent_uuid: "53e9fb42a73442ffa9babcbcdf8587f9" member_type: VOTER last_known_addr { host: "127.19.96.124" port: 37201 } }
I20260812 08:03:12.129179 20747 raft_consensus.cc:385] T 00000000000000000000000000000000 P 8176988cf19f4bb199c5891c11aade49 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 08:03:12.129226 20747 raft_consensus.cc:740] T 00000000000000000000000000000000 P 8176988cf19f4bb199c5891c11aade49 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8176988cf19f4bb199c5891c11aade49, State: Initialized, Role: FOLLOWER
I20260812 08:03:12.129390 20747 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8176988cf19f4bb199c5891c11aade49 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9be450af28cb4085992ad5e0ebb480b9" member_type: VOTER last_known_addr { host: "127.19.96.126" port: 36207 } } peers { permanent_uuid: "8176988cf19f4bb199c5891c11aade49" member_type: VOTER last_known_addr { host: "127.19.96.125" port: 40969 } } peers { permanent_uuid: "53e9fb42a73442ffa9babcbcdf8587f9" member_type: VOTER last_known_addr { host: "127.19.96.124" port: 37201 } }
I20260812 08:03:12.129743 20862 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8176988cf19f4bb199c5891c11aade49 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9be450af28cb4085992ad5e0ebb480b9" member_type: VOTER last_known_addr { host: "127.19.96.126" port: 36207 } } peers { permanent_uuid: "8176988cf19f4bb199c5891c11aade49" member_type: VOTER last_known_addr { host: "127.19.96.125" port: 40969 } } peers { permanent_uuid: "53e9fb42a73442ffa9babcbcdf8587f9" member_type: VOTER last_known_addr { host: "127.19.96.124" port: 37201 } } }
I20260812 08:03:12.129828 20862 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8176988cf19f4bb199c5891c11aade49 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 08:03:12.129951 20747 sys_catalog.cc:565] T 00000000000000000000000000000000 P 8176988cf19f4bb199c5891c11aade49 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 08:03:12.131664 19841 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 2
I20260812 08:03:12.132243 19841 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 08:03:12.132428 20875 catalog_manager.cc:1594] T 00000000000000000000000000000000 P 8176988cf19f4bb199c5891c11aade49: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 08:03:12.132493 20875 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260812 08:03:12.134354 20876 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:03:12.134446 19841 server_base.cc:1061] running on GCE node
W20260812 08:03:12.134639 20882 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:03:12.134461 20878 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:03:12.134958 19841 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260812 08:03:12.135038 19841 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260812 08:03:12.135064 19841 hybrid_clock.cc:648] HybridClock initialized: now 1786521792135063 us; error 0 us; skew 500 ppm
I20260812 08:03:12.136202 19841 webserver.cc:533] Webserver started at http://127.19.96.65:38587/ using document root <none> and password file <none>
I20260812 08:03:12.136407 19841 fs_manager.cc:362] Metadata directory not provided
I20260812 08:03:12.136489 19841 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 08:03:12.136590 19841 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 08:03:12.137075 19841 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task3yToRy/test-tmp/prometheus_sd-itest.0.MultiMasterPrometheusSDITest.LeaderFollowerSDBehavior.1786521770778185-19841-0/minicluster-data/ts-0-root/instance:
uuid: "b4be88ca1fad415c9db7d2f7ef44bdda"
format_stamp: "Formatted at 2026-08-12 08:03:12 on dist-test-slave-8w9v"
I20260812 08:03:12.138685 19841 fs_manager.cc:696] Time spent creating directory manager: real 0.001s	user 0.002s	sys 0.000s
I20260812 08:03:12.139693 20889 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:03:12.139977 19841 fs_manager.cc:730] Time spent opening block manager: real 0.001s	user 0.001s	sys 0.000s
I20260812 08:03:12.140093 19841 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task3yToRy/test-tmp/prometheus_sd-itest.0.MultiMasterPrometheusSDITest.LeaderFollowerSDBehavior.1786521770778185-19841-0/minicluster-data/ts-0-root
uuid: "b4be88ca1fad415c9db7d2f7ef44bdda"
format_stamp: "Formatted at 2026-08-12 08:03:12 on dist-test-slave-8w9v"
I20260812 08:03:12.140195 19841 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task3yToRy/test-tmp/prometheus_sd-itest.0.MultiMasterPrometheusSDITest.LeaderFollowerSDBehavior.1786521770778185-19841-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-task3yToRy/test-tmp/prometheus_sd-itest.0.MultiMasterPrometheusSDITest.LeaderFollowerSDBehavior.1786521770778185-19841-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-task3yToRy/test-tmp/prometheus_sd-itest.0.MultiMasterPrometheusSDITest.LeaderFollowerSDBehavior.1786521770778185-19841-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 08:03:12.151369 19841 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:03:12.151849 19841 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:03:12.152280 19841 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260812 08:03:12.152872 19841 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 08:03:12.152944 19841 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:03:12.153007 19841 ts_tablet_manager.cc:616] Registered 0 tablets
I20260812 08:03:12.153064 19841 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:03:12.159142 19841 rpc_server.cc:307] RPC server started. Bound to: 127.19.96.65:35041
I20260812 08:03:12.159188 20965 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.19.96.65:35041 every 8 connection(s)
I20260812 08:03:12.165269 20967 heartbeater.cc:344] Connected to a master server at 127.19.96.126:36207
I20260812 08:03:12.165413 20967 heartbeater.cc:461] Registering TS with master...
I20260812 08:03:12.165727 20967 heartbeater.cc:507] Master 127.19.96.126:36207 requested a full tablet report, sending...
I20260812 08:03:12.165870 20968 heartbeater.cc:344] Connected to a master server at 127.19.96.125:40969
I20260812 08:03:12.166036 20968 heartbeater.cc:461] Registering TS with master...
I20260812 08:03:12.166280 20968 heartbeater.cc:507] Master 127.19.96.125:40969 requested a full tablet report, sending...
I20260812 08:03:12.166585 20966 heartbeater.cc:344] Connected to a master server at 127.19.96.124:37201
I20260812 08:03:12.166604 20614 ts_manager.cc:194] Registered new tserver with Master: b4be88ca1fad415c9db7d2f7ef44bdda (127.19.96.65:35041)
I20260812 08:03:12.166715 20966 heartbeater.cc:461] Registering TS with master...
I20260812 08:03:12.166939 20966 heartbeater.cc:507] Master 127.19.96.124:37201 requested a full tablet report, sending...
I20260812 08:03:12.167075 20699 ts_manager.cc:194] Registered new tserver with Master: b4be88ca1fad415c9db7d2f7ef44bdda (127.19.96.65:35041)
I20260812 08:03:12.167637 20782 ts_manager.cc:194] Registered new tserver with Master: b4be88ca1fad415c9db7d2f7ef44bdda (127.19.96.65:35041)
I20260812 08:03:12.168691 19841 internal_mini_cluster.cc:371] 1 TS(s) registered with all masters after 0.00887302s
I20260812 08:03:12.169325 19841 internal_mini_cluster.cc:371] 1 TS(s) registered with all masters after 4.5486e-05s
I20260812 08:03:12.196614 20846 raft_consensus.cc:493] T 00000000000000000000000000000000 P 9be450af28cb4085992ad5e0ebb480b9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 08:03:12.197199 20846 raft_consensus.cc:515] T 00000000000000000000000000000000 P 9be450af28cb4085992ad5e0ebb480b9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9be450af28cb4085992ad5e0ebb480b9" member_type: VOTER last_known_addr { host: "127.19.96.126" port: 36207 } } peers { permanent_uuid: "8176988cf19f4bb199c5891c11aade49" member_type: VOTER last_known_addr { host: "127.19.96.125" port: 40969 } } peers { permanent_uuid: "53e9fb42a73442ffa9babcbcdf8587f9" member_type: VOTER last_known_addr { host: "127.19.96.124" port: 37201 } }
I20260812 08:03:12.199275 20846 leader_election.cc:290] T 00000000000000000000000000000000 P 9be450af28cb4085992ad5e0ebb480b9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8176988cf19f4bb199c5891c11aade49 (127.19.96.125:40969), 53e9fb42a73442ffa9babcbcdf8587f9 (127.19.96.124:37201)
I20260812 08:03:12.200917 20793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "9be450af28cb4085992ad5e0ebb480b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "53e9fb42a73442ffa9babcbcdf8587f9" is_pre_election: true
I20260812 08:03:12.200924 20710 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "9be450af28cb4085992ad5e0ebb480b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8176988cf19f4bb199c5891c11aade49" is_pre_election: true
I20260812 08:03:12.204295 20710 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 8176988cf19f4bb199c5891c11aade49 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9be450af28cb4085992ad5e0ebb480b9 in term 0.
I20260812 08:03:12.205024 20598 leader_election.cc:304] T 00000000000000000000000000000000 P 9be450af28cb4085992ad5e0ebb480b9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8176988cf19f4bb199c5891c11aade49, 9be450af28cb4085992ad5e0ebb480b9; no voters: 
I20260812 08:03:12.205293 20846 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 9be450af28cb4085992ad5e0ebb480b9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 08:03:12.205344 20846 raft_consensus.cc:493] T 00000000000000000000000000000000 P 9be450af28cb4085992ad5e0ebb480b9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 08:03:12.205379 20846 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 9be450af28cb4085992ad5e0ebb480b9 [term 0 FOLLOWER]: Advancing to term 1
I20260812 08:03:12.206806 20846 raft_consensus.cc:515] T 00000000000000000000000000000000 P 9be450af28cb4085992ad5e0ebb480b9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9be450af28cb4085992ad5e0ebb480b9" member_type: VOTER last_known_addr { host: "127.19.96.126" port: 36207 } } peers { permanent_uuid: "8176988cf19f4bb199c5891c11aade49" member_type: VOTER last_known_addr { host: "127.19.96.125" port: 40969 } } peers { permanent_uuid: "53e9fb42a73442ffa9babcbcdf8587f9" member_type: VOTER last_known_addr { host: "127.19.96.124" port: 37201 } }
I20260812 08:03:12.207002 20793 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 53e9fb42a73442ffa9babcbcdf8587f9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9be450af28cb4085992ad5e0ebb480b9 in term 0.
I20260812 08:03:12.207705 20709 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "9be450af28cb4085992ad5e0ebb480b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8176988cf19f4bb199c5891c11aade49"
I20260812 08:03:12.207840 20709 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 8176988cf19f4bb199c5891c11aade49 [term 0 FOLLOWER]: Advancing to term 1
I20260812 08:03:12.208693 20846 leader_election.cc:290] T 00000000000000000000000000000000 P 9be450af28cb4085992ad5e0ebb480b9 [CANDIDATE]: Term 1 election: Requested vote from peers 8176988cf19f4bb199c5891c11aade49 (127.19.96.125:40969), 53e9fb42a73442ffa9babcbcdf8587f9 (127.19.96.124:37201)
I20260812 08:03:12.209102 20709 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 8176988cf19f4bb199c5891c11aade49 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9be450af28cb4085992ad5e0ebb480b9 in term 1.
I20260812 08:03:12.209357 20793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "9be450af28cb4085992ad5e0ebb480b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "53e9fb42a73442ffa9babcbcdf8587f9"
I20260812 08:03:12.209523 20598 leader_election.cc:304] T 00000000000000000000000000000000 P 9be450af28cb4085992ad5e0ebb480b9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8176988cf19f4bb199c5891c11aade49, 9be450af28cb4085992ad5e0ebb480b9; no voters: 
I20260812 08:03:12.209546 20793 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 53e9fb42a73442ffa9babcbcdf8587f9 [term 0 FOLLOWER]: Advancing to term 1
I20260812 08:03:12.209756 20846 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 9be450af28cb4085992ad5e0ebb480b9 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 08:03:12.210044 20846 raft_consensus.cc:697] T 00000000000000000000000000000000 P 9be450af28cb4085992ad5e0ebb480b9 [term 1 LEADER]: Becoming Leader. State: Replica: 9be450af28cb4085992ad5e0ebb480b9, State: Running, Role: LEADER
I20260812 08:03:12.210271 20846 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9be450af28cb4085992ad5e0ebb480b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9be450af28cb4085992ad5e0ebb480b9" member_type: VOTER last_known_addr { host: "127.19.96.126" port: 36207 } } peers { permanent_uuid: "8176988cf19f4bb199c5891c11aade49" member_type: VOTER last_known_addr { host: "127.19.96.125" port: 40969 } } peers { permanent_uuid: "53e9fb42a73442ffa9babcbcdf8587f9" member_type: VOTER last_known_addr { host: "127.19.96.124" port: 37201 } }
I20260812 08:03:12.211572 20793 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 53e9fb42a73442ffa9babcbcdf8587f9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9be450af28cb4085992ad5e0ebb480b9 in term 1.
I20260812 08:03:12.211831 20846 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9be450af28cb4085992ad5e0ebb480b9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9be450af28cb4085992ad5e0ebb480b9. Latest consensus state: current_term: 1 leader_uuid: "9be450af28cb4085992ad5e0ebb480b9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9be450af28cb4085992ad5e0ebb480b9" member_type: VOTER last_known_addr { host: "127.19.96.126" port: 36207 } } peers { permanent_uuid: "8176988cf19f4bb199c5891c11aade49" member_type: VOTER last_known_addr { host: "127.19.96.125" port: 40969 } } peers { permanent_uuid: "53e9fb42a73442ffa9babcbcdf8587f9" member_type: VOTER last_known_addr { host: "127.19.96.124" port: 37201 } } }
I20260812 08:03:12.211966 20846 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9be450af28cb4085992ad5e0ebb480b9 [sys.catalog]: This master's current role is: LEADER
I20260812 08:03:12.212422 20989 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260812 08:03:12.213266 20989 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260812 08:03:12.216673 20709 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 8176988cf19f4bb199c5891c11aade49 [term 1 FOLLOWER]: Refusing update from remote peer 9be450af28cb4085992ad5e0ebb480b9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 08:03:12.217257 20987 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 9be450af28cb4085992ad5e0ebb480b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8176988cf19f4bb199c5891c11aade49" member_type: VOTER last_known_addr { host: "127.19.96.125" port: 40969 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 08:03:12.219204 20793 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 53e9fb42a73442ffa9babcbcdf8587f9 [term 1 FOLLOWER]: Refusing update from remote peer 9be450af28cb4085992ad5e0ebb480b9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 08:03:12.219532 20862 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8176988cf19f4bb199c5891c11aade49 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9be450af28cb4085992ad5e0ebb480b9. Latest consensus state: current_term: 1 leader_uuid: "9be450af28cb4085992ad5e0ebb480b9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9be450af28cb4085992ad5e0ebb480b9" member_type: VOTER last_known_addr { host: "127.19.96.126" port: 36207 } } peers { permanent_uuid: "8176988cf19f4bb199c5891c11aade49" member_type: VOTER last_known_addr { host: "127.19.96.125" port: 40969 } } peers { permanent_uuid: "53e9fb42a73442ffa9babcbcdf8587f9" member_type: VOTER last_known_addr { host: "127.19.96.124" port: 37201 } } }
I20260812 08:03:12.219636 20862 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8176988cf19f4bb199c5891c11aade49 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 08:03:12.219941 20987 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 9be450af28cb4085992ad5e0ebb480b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "53e9fb42a73442ffa9babcbcdf8587f9" member_type: VOTER last_known_addr { host: "127.19.96.124" port: 37201 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 08:03:12.222530 20831 sys_catalog.cc:455] T 00000000000000000000000000000000 P 53e9fb42a73442ffa9babcbcdf8587f9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9be450af28cb4085992ad5e0ebb480b9. Latest consensus state: current_term: 1 leader_uuid: "9be450af28cb4085992ad5e0ebb480b9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9be450af28cb4085992ad5e0ebb480b9" member_type: VOTER last_known_addr { host: "127.19.96.126" port: 36207 } } peers { permanent_uuid: "8176988cf19f4bb199c5891c11aade49" member_type: VOTER last_known_addr { host: "127.19.96.125" port: 40969 } } peers { permanent_uuid: "53e9fb42a73442ffa9babcbcdf8587f9" member_type: VOTER last_known_addr { host: "127.19.96.124" port: 37201 } } }
I20260812 08:03:12.222693 20831 sys_catalog.cc:458] T 00000000000000000000000000000000 P 53e9fb42a73442ffa9babcbcdf8587f9 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 08:03:12.224097 20862 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8176988cf19f4bb199c5891c11aade49 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "9be450af28cb4085992ad5e0ebb480b9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9be450af28cb4085992ad5e0ebb480b9" member_type: VOTER last_known_addr { host: "127.19.96.126" port: 36207 } } peers { permanent_uuid: "8176988cf19f4bb199c5891c11aade49" member_type: VOTER last_known_addr { host: "127.19.96.125" port: 40969 } } peers { permanent_uuid: "53e9fb42a73442ffa9babcbcdf8587f9" member_type: VOTER last_known_addr { host: "127.19.96.124" port: 37201 } } }
I20260812 08:03:12.224247 20862 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8176988cf19f4bb199c5891c11aade49 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 08:03:12.224545 20831 sys_catalog.cc:455] T 00000000000000000000000000000000 P 53e9fb42a73442ffa9babcbcdf8587f9 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "9be450af28cb4085992ad5e0ebb480b9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9be450af28cb4085992ad5e0ebb480b9" member_type: VOTER last_known_addr { host: "127.19.96.126" port: 36207 } } peers { permanent_uuid: "8176988cf19f4bb199c5891c11aade49" member_type: VOTER last_known_addr { host: "127.19.96.125" port: 40969 } } peers { permanent_uuid: "53e9fb42a73442ffa9babcbcdf8587f9" member_type: VOTER last_known_addr { host: "127.19.96.124" port: 37201 } } }
I20260812 08:03:12.224710 20831 sys_catalog.cc:458] T 00000000000000000000000000000000 P 53e9fb42a73442ffa9babcbcdf8587f9 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 08:03:12.224748 20993 mvcc.cc:204] Tried to move back new op lower bound from 7317593260910669824 to 7317593260898054144. Current Snapshot: MvccSnapshot[applied={T|T < 7317593260910669824}]
I20260812 08:03:12.225816 20846 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9be450af28cb4085992ad5e0ebb480b9 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "9be450af28cb4085992ad5e0ebb480b9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9be450af28cb4085992ad5e0ebb480b9" member_type: VOTER last_known_addr { host: "127.19.96.126" port: 36207 } } peers { permanent_uuid: "8176988cf19f4bb199c5891c11aade49" member_type: VOTER last_known_addr { host: "127.19.96.125" port: 40969 } } peers { permanent_uuid: "53e9fb42a73442ffa9babcbcdf8587f9" member_type: VOTER last_known_addr { host: "127.19.96.124" port: 37201 } } }
I20260812 08:03:12.225986 20846 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9be450af28cb4085992ad5e0ebb480b9 [sys.catalog]: This master's current role is: LEADER
I20260812 08:03:12.226171 20846 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9be450af28cb4085992ad5e0ebb480b9 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "9be450af28cb4085992ad5e0ebb480b9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9be450af28cb4085992ad5e0ebb480b9" member_type: VOTER last_known_addr { host: "127.19.96.126" port: 36207 } } peers { permanent_uuid: "8176988cf19f4bb199c5891c11aade49" member_type: VOTER last_known_addr { host: "127.19.96.125" port: 40969 } } peers { permanent_uuid: "53e9fb42a73442ffa9babcbcdf8587f9" member_type: VOTER last_known_addr { host: "127.19.96.124" port: 37201 } } }
I20260812 08:03:12.226274 20846 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9be450af28cb4085992ad5e0ebb480b9 [sys.catalog]: This master's current role is: LEADER
I20260812 08:03:12.227094 20989 catalog_manager.cc:1383] Generated new cluster ID: c1cc5bc8d3b24e01a31bbb1c39519648
I20260812 08:03:12.227154 20989 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260812 08:03:12.253202 20989 catalog_manager.cc:1406] Generated new certificate authority record
I20260812 08:03:12.254453 20989 catalog_manager.cc:1540] Loading token signing keys...
I20260812 08:03:12.270231 20989 catalog_manager.cc:6092] T 00000000000000000000000000000000 P 9be450af28cb4085992ad5e0ebb480b9: Generated new TSK 0
I20260812 08:03:12.270539 20989 catalog_manager.cc:1550] Initializing in-progress tserver states...
time=2026-08-12T08:03:12.266Z level=INFO source=main.go:1678 msg="updated GOGC" old=100 new=75
time=2026-08-12T08:03:12.272Z level=INFO source=main.go:744 msg="Leaving GOMAXPROCS=4: CPU quota undefined" component=automaxprocs
time=2026-08-12T08:03:12.273Z level=INFO source=memlimit.go:198 msg="GOMEMLIMIT is updated" component=automemlimit package=github.com/KimMachineGun/automemlimit/memlimit GOMEMLIMIT=14142552883 previous=9223372036854775807
time=2026-08-12T08:03:12.273Z level=INFO source=main.go:851 msg="Starting Prometheus Server" mode=server version="(version=3.11.2, branch=HEAD, revision=f0f0fdd679dcd6df320b0558b20919f7cd44c407)"
time=2026-08-12T08:03:12.273Z level=INFO source=main.go:856 msg="operational information" build_context="(go=go1.26.2, platform=linux/amd64, user=root@d2684485c347, date=20260413-11:53:40, tags=netgo,builtinassets)" host_details="(Linux 5.4.0-1157-gcp #166~18.04.1-Ubuntu SMP Wed Jan 21 01:57:46 UTC 2026 x86_64 dist-test-slave-8w9v (none))" fd_limits="(soft=1048576, hard=1048576)" vm_limits="(soft=unlimited, hard=unlimited)"
time=2026-08-12T08:03:12.522Z level=INFO source=web.go:710 msg="Start listening for connections" component=web address=127.0.0.1:0
time=2026-08-12T08:03:12.523Z level=INFO source=main.go:1410 msg="Starting TSDB ..."
time=2026-08-12T08:03:12.527Z level=INFO source=tls_config.go:354 msg="Listening on" component=web address=127.0.0.1:43581
time=2026-08-12T08:03:12.528Z level=INFO source=tls_config.go:357 msg="TLS is disabled." component=web http2=false address=127.0.0.1:43581
time=2026-08-12T08:03:12.528Z level=INFO source=head.go:698 msg="Replaying on-disk memory mappable chunks if any" component=tsdb
time=2026-08-12T08:03:12.528Z level=INFO source=head.go:784 msg="On-disk memory mappable chunks replay completed" component=tsdb duration=2.741µs
time=2026-08-12T08:03:12.528Z level=INFO source=head.go:792 msg="Replaying WAL, this may take a while" component=tsdb
time=2026-08-12T08:03:12.529Z level=INFO source=head.go:865 msg="WAL segment loaded" component=tsdb segment=0 maxSegment=0 duration=382.873µs
time=2026-08-12T08:03:12.529Z level=INFO source=head.go:902 msg="WAL replay completed" component=tsdb checkpoint_replay_duration=62.367µs wal_replay_duration=419.386µs wbl_replay_duration=220ns chunk_snapshot_load_duration=0s mmap_chunk_replay_duration=2.741µs total_replay_duration=543.98µs
time=2026-08-12T08:03:12.531Z level=INFO source=main.go:1431 msg="filesystem information" fs_type=OVERLAYFS_SUPER_MAGIC
time=2026-08-12T08:03:12.531Z level=INFO source=main.go:1434 msg="TSDB started"
time=2026-08-12T08:03:12.531Z level=INFO source=main.go:1632 msg="Loading configuration file" filename=/tmp/dist-test-task3yToRy/test-tmp/prometheus_sd-itest.0.MultiMasterPrometheusSDITest.LeaderFollowerSDBehavior.1786521770778185-19841-0/prometheus.yml
time=2026-08-12T08:03:12.531Z level=INFO source=main.go:1048 msg="TSDB retention updated" duration=15d size=0B percentage=0
time=2026-08-12T08:03:12.533Z level=INFO source=main.go:1671 msg="Completed loading of configuration file" db_storage=59.275µs remote_storage=2.464µs web_handler=447ns query_engine=2.293µs scrape=1.885134ms scrape_sd=262.577µs notify=2.552µs notify_sd=1.086µs rules=2.705µs tracing=5.882µs filename=/tmp/dist-test-task3yToRy/test-tmp/prometheus_sd-itest.0.MultiMasterPrometheusSDITest.LeaderFollowerSDBehavior.1786521770778185-19841-0/prometheus.yml totalDuration=2.570946ms
time=2026-08-12T08:03:12.534Z level=INFO source=main.go:1395 msg="Server is ready to receive web requests."
time=2026-08-12T08:03:12.534Z level=INFO source=manager.go:209 msg="Starting rule manager..." component="rule manager"
I20260812 08:03:13.100680 20843 catalog_manager.cc:1295] Loaded cluster ID: c1cc5bc8d3b24e01a31bbb1c39519648
I20260812 08:03:13.100803 20843 catalog_manager.cc:1588] T 00000000000000000000000000000000 P 53e9fb42a73442ffa9babcbcdf8587f9: loading cluster ID for follower catalog manager: success
I20260812 08:03:13.102569 20843 catalog_manager.cc:1610] T 00000000000000000000000000000000 P 53e9fb42a73442ffa9babcbcdf8587f9: acquiring CA information for follower catalog manager: success
I20260812 08:03:13.103425 20843 catalog_manager.cc:1638] T 00000000000000000000000000000000 P 53e9fb42a73442ffa9babcbcdf8587f9: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260812 08:03:13.133431 20875 catalog_manager.cc:1295] Loaded cluster ID: c1cc5bc8d3b24e01a31bbb1c39519648
I20260812 08:03:13.133531 20875 catalog_manager.cc:1588] T 00000000000000000000000000000000 P 8176988cf19f4bb199c5891c11aade49: loading cluster ID for follower catalog manager: success
I20260812 08:03:13.134771 20875 catalog_manager.cc:1610] T 00000000000000000000000000000000 P 8176988cf19f4bb199c5891c11aade49: acquiring CA information for follower catalog manager: success
I20260812 08:03:13.135432 20875 catalog_manager.cc:1638] T 00000000000000000000000000000000 P 8176988cf19f4bb199c5891c11aade49: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260812 08:03:13.169302 20614 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:51200
I20260812 08:03:14.170401 20967 heartbeater.cc:499] Master 127.19.96.126:36207 was elected leader, sending a full tablet report...
time=2026-08-12T08:03:19.348Z level=WARN source=main.go:1191 msg="Received an OS signal, exiting gracefully..." signal=terminated
time=2026-08-12T08:03:19.348Z level=INFO source=main.go:1216 msg="Stopping scrape discovery manager..."
time=2026-08-12T08:03:19.348Z level=INFO source=main.go:1230 msg="Stopping notify discovery manager..."
time=2026-08-12T08:03:19.348Z level=INFO source=main.go:1245 msg="Stopping rule manager manager..."
time=2026-08-12T08:03:19.348Z level=INFO source=manager.go:223 msg="Stopping rule manager..." component="rule manager"
time=2026-08-12T08:03:19.348Z level=INFO source=manager.go:239 msg="Rule manager stopped" component="rule manager"
time=2026-08-12T08:03:19.348Z level=INFO source=main.go:1269 msg="Stopping scrape manager..."
time=2026-08-12T08:03:19.359Z level=INFO source=main.go:1226 msg="Notify discovery manager stopped"
time=2026-08-12T08:03:19.359Z level=INFO source=main.go:1212 msg="Scrape discovery manager stopped"
time=2026-08-12T08:03:19.359Z level=INFO source=main.go:1241 msg="Rule manager stopped"
time=2026-08-12T08:03:19.359Z level=INFO source=main.go:1284 msg="Stopping tracing manager..."
time=2026-08-12T08:03:19.359Z level=INFO source=main.go:1261 msg="Scrape manager stopped"
time=2026-08-12T08:03:19.359Z level=INFO source=main.go:1369 msg="Stopping reloaders..."
time=2026-08-12T08:03:19.359Z level=INFO source=main.go:1458 msg="Stopping storage..."
time=2026-08-12T08:03:19.359Z level=INFO source=main.go:1280 msg="Tracing manager stopped"
time=2026-08-12T08:03:19.359Z level=INFO source=main.go:1361 msg="Reloaders stopped"
time=2026-08-12T08:03:19.364Z level=INFO source=main.go:1536 msg="Stopping web handler..."
time=2026-08-12T08:03:19.364Z level=INFO source=main.go:1559 msg="Stopping notifier manager..."
time=2026-08-12T08:03:19.364Z level=INFO source=manager.go:321 msg="Stopping notification manager..." component=notifier
time=2026-08-12T08:03:19.364Z level=INFO source=main.go:1454 msg="TSDB stopped"
time=2026-08-12T08:03:19.364Z level=INFO source=main.go:1555 msg="Notifier manager stopped"
time=2026-08-12T08:03:19.365Z level=INFO source=main.go:1532 msg="Web handler stopped"
time=2026-08-12T08:03:19.365Z level=INFO source=main.go:1570 msg="See you next time!"
I20260812 08:03:19.379515 19841 tablet_server.cc:179] TabletServer@127.19.96.65:0 shutting down...
I20260812 08:03:19.384203 19841 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260812 08:03:19.396989 19841 tablet_server.cc:196] TabletServer@127.19.96.65:0 shutdown complete.
I20260812 08:03:19.399403 19841 master.cc:562] Master@127.19.96.126:36207 shutting down...
I20260812 08:03:19.403891 19841 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 9be450af28cb4085992ad5e0ebb480b9 [term 1 LEADER]: Raft consensus shutting down.
I20260812 08:03:19.404369 19841 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 9be450af28cb4085992ad5e0ebb480b9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 08:03:19.404448 19841 tablet_replica.cc:333] T 00000000000000000000000000000000 P 9be450af28cb4085992ad5e0ebb480b9: stopping tablet replica
I20260812 08:03:19.417740 19841 master.cc:584] Master@127.19.96.126:36207 shutdown complete.
I20260812 08:03:19.420934 19841 master.cc:562] Master@127.19.96.125:40969 shutting down...
I20260812 08:03:19.425920 19841 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 8176988cf19f4bb199c5891c11aade49 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 08:03:19.426133 19841 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 8176988cf19f4bb199c5891c11aade49 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 08:03:19.426187 19841 tablet_replica.cc:333] T 00000000000000000000000000000000 P 8176988cf19f4bb199c5891c11aade49: stopping tablet replica
I20260812 08:03:19.440356 19841 master.cc:584] Master@127.19.96.125:40969 shutdown complete.
I20260812 08:03:19.443176 19841 master.cc:562] Master@127.19.96.124:37201 shutting down...
I20260812 08:03:19.447486 19841 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 53e9fb42a73442ffa9babcbcdf8587f9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 08:03:19.447716 19841 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 53e9fb42a73442ffa9babcbcdf8587f9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 08:03:19.447808 19841 tablet_replica.cc:333] T 00000000000000000000000000000000 P 53e9fb42a73442ffa9babcbcdf8587f9: stopping tablet replica
I20260812 08:03:19.460917 19841 master.cc:584] Master@127.19.96.124:37201 shutdown complete.
[       OK ] MultiMasterPrometheusSDITest.LeaderFollowerSDBehavior (7464 ms)
[ RUN      ] MultiMasterPrometheusSDITest.EmptyArrayBehaviorOnFollower
I20260812 08:03:19.467672 19841 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.19.96.126:40893,127.19.96.125:41391,127.19.96.124:45949
I20260812 08:03:19.468274 19841 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 08:03:19.470325 21021 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:03:19.470336 21019 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:03:19.470341 21018 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:03:19.470641 19841 server_base.cc:1061] running on GCE node
I20260812 08:03:19.470876 19841 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260812 08:03:19.470921 19841 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260812 08:03:19.470937 19841 hybrid_clock.cc:648] HybridClock initialized: now 1786521799470937 us; error 0 us; skew 500 ppm
I20260812 08:03:19.471762 19841 webserver.cc:533] Webserver started at http://127.19.96.126:40813/ using document root <none> and password file <none>
I20260812 08:03:19.471930 19841 fs_manager.cc:362] Metadata directory not provided
I20260812 08:03:19.471976 19841 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 08:03:19.472040 19841 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 08:03:19.472520 19841 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task3yToRy/test-tmp/prometheus_sd-itest.0.MultiMasterPrometheusSDITest.EmptyArrayBehaviorOnFollower.1786521770778185-19841-0/minicluster-data/master-0-root/instance:
uuid: "05617cbaf05f4626a0992dae988ce9f9"
format_stamp: "Formatted at 2026-08-12 08:03:19 on dist-test-slave-8w9v"
I20260812 08:03:19.474025 19841 fs_manager.cc:696] Time spent creating directory manager: real 0.001s	user 0.000s	sys 0.002s
I20260812 08:03:19.475028 21028 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:03:19.475391 19841 fs_manager.cc:730] Time spent opening block manager: real 0.001s	user 0.000s	sys 0.001s
I20260812 08:03:19.475503 19841 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task3yToRy/test-tmp/prometheus_sd-itest.0.MultiMasterPrometheusSDITest.EmptyArrayBehaviorOnFollower.1786521770778185-19841-0/minicluster-data/master-0-root
uuid: "05617cbaf05f4626a0992dae988ce9f9"
format_stamp: "Formatted at 2026-08-12 08:03:19 on dist-test-slave-8w9v"
I20260812 08:03:19.475596 19841 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task3yToRy/test-tmp/prometheus_sd-itest.0.MultiMasterPrometheusSDITest.EmptyArrayBehaviorOnFollower.1786521770778185-19841-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-task3yToRy/test-tmp/prometheus_sd-itest.0.MultiMasterPrometheusSDITest.EmptyArrayBehaviorOnFollower.1786521770778185-19841-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-task3yToRy/test-tmp/prometheus_sd-itest.0.MultiMasterPrometheusSDITest.EmptyArrayBehaviorOnFollower.1786521770778185-19841-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 08:03:19.494114 19841 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:03:19.494616 19841 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:03:19.499918 19841 rpc_server.cc:307] RPC server started. Bound to: 127.19.96.126:40893
I20260812 08:03:19.501642 19841 file_cache.cc:504] Constructed file cache file cache with capacity 419430
I20260812 08:03:19.503600 21088 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.19.96.126:40893 every 8 connection(s)
I20260812 08:03:19.504510 21089 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
W20260812 08:03:19.504706 21091 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:03:19.504741 21092 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:03:19.504882 19841 server_base.cc:1061] running on GCE node
I20260812 08:03:19.506040 21089 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.96.126" port: 40893 } has no permanent_uuid. Determining permanent_uuid...
W20260812 08:03:19.507180 21094 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:03:19.507473 19841 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260812 08:03:19.507541 19841 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260812 08:03:19.507565 19841 hybrid_clock.cc:648] HybridClock initialized: now 1786521799507564 us; error 0 us; skew 500 ppm
I20260812 08:03:19.508574 19841 webserver.cc:533] Webserver started at http://127.19.96.125:36859/ using document root <none> and password file <none>
I20260812 08:03:19.508733 19841 fs_manager.cc:362] Metadata directory not provided
I20260812 08:03:19.508787 19841 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 08:03:19.508865 19841 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 08:03:19.509276 19841 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task3yToRy/test-tmp/prometheus_sd-itest.0.MultiMasterPrometheusSDITest.EmptyArrayBehaviorOnFollower.1786521770778185-19841-0/minicluster-data/master-1-root/instance:
uuid: "3d462115eb1741df9deb19ba42ac53aa"
format_stamp: "Formatted at 2026-08-12 08:03:19 on dist-test-slave-8w9v"
I20260812 08:03:19.511072 19841 fs_manager.cc:696] Time spent creating directory manager: real 0.002s	user 0.002s	sys 0.000s
I20260812 08:03:19.512207 21103 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:03:19.512486 19841 fs_manager.cc:730] Time spent opening block manager: real 0.001s	user 0.001s	sys 0.000s
I20260812 08:03:19.512584 19841 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task3yToRy/test-tmp/prometheus_sd-itest.0.MultiMasterPrometheusSDITest.EmptyArrayBehaviorOnFollower.1786521770778185-19841-0/minicluster-data/master-1-root
uuid: "3d462115eb1741df9deb19ba42ac53aa"
format_stamp: "Formatted at 2026-08-12 08:03:19 on dist-test-slave-8w9v"
I20260812 08:03:19.512678 19841 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task3yToRy/test-tmp/prometheus_sd-itest.0.MultiMasterPrometheusSDITest.EmptyArrayBehaviorOnFollower.1786521770778185-19841-0/minicluster-data/master-1-root
metadata directory: /tmp/dist-test-task3yToRy/test-tmp/prometheus_sd-itest.0.MultiMasterPrometheusSDITest.EmptyArrayBehaviorOnFollower.1786521770778185-19841-0/minicluster-data/master-1-root
1 data directories: /tmp/dist-test-task3yToRy/test-tmp/prometheus_sd-itest.0.MultiMasterPrometheusSDITest.EmptyArrayBehaviorOnFollower.1786521770778185-19841-0/minicluster-data/master-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 08:03:19.517217 21089 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.96.125" port: 41391 } has no permanent_uuid. Determining permanent_uuid...
W20260812 08:03:19.518033 21033 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.19.96.125:41391: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260812 08:03:19.518958 21089 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.19.96.125:41391: Network error: Client connection negotiation failed: client connection to 127.19.96.125:41391: connect: Connection refused (error 111)
I20260812 08:03:19.539011 19841 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:03:19.539453 19841 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:03:19.544188 19841 rpc_server.cc:307] RPC server started. Bound to: 127.19.96.125:41391
I20260812 08:03:19.544207 21170 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.19.96.125:41391 every 8 connection(s)
I20260812 08:03:19.545044 19841 file_cache.cc:504] Constructed file cache file cache with capacity 419430
I20260812 08:03:19.545250 21171 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
W20260812 08:03:19.546871 21175 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:03:19.546999 21173 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:03:19.547088 21171 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.96.126" port: 40893 } has no permanent_uuid. Determining permanent_uuid...
I20260812 08:03:19.547209 19841 server_base.cc:1061] running on GCE node
W20260812 08:03:19.547297 21177 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:03:19.547554 19841 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260812 08:03:19.547628 19841 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260812 08:03:19.547662 19841 hybrid_clock.cc:648] HybridClock initialized: now 1786521799547660 us; error 0 us; skew 500 ppm
I20260812 08:03:19.548667 19841 webserver.cc:533] Webserver started at http://127.19.96.124:42067/ using document root <none> and password file <none>
I20260812 08:03:19.548867 19841 fs_manager.cc:362] Metadata directory not provided
I20260812 08:03:19.548943 19841 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 08:03:19.549022 19841 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 08:03:19.549379 19841 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task3yToRy/test-tmp/prometheus_sd-itest.0.MultiMasterPrometheusSDITest.EmptyArrayBehaviorOnFollower.1786521770778185-19841-0/minicluster-data/master-2-root/instance:
uuid: "6d5c9a7153884097989bf782fd594c07"
format_stamp: "Formatted at 2026-08-12 08:03:19 on dist-test-slave-8w9v"
I20260812 08:03:19.551433 19841 fs_manager.cc:696] Time spent creating directory manager: real 0.002s	user 0.002s	sys 0.000s
I20260812 08:03:19.551898 21171 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.96.125" port: 41391 } has no permanent_uuid. Determining permanent_uuid...
I20260812 08:03:19.552870 21185 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:03:19.553145 19841 fs_manager.cc:730] Time spent opening block manager: real 0.001s	user 0.000s	sys 0.000s
I20260812 08:03:19.553257 19841 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task3yToRy/test-tmp/prometheus_sd-itest.0.MultiMasterPrometheusSDITest.EmptyArrayBehaviorOnFollower.1786521770778185-19841-0/minicluster-data/master-2-root
uuid: "6d5c9a7153884097989bf782fd594c07"
format_stamp: "Formatted at 2026-08-12 08:03:19 on dist-test-slave-8w9v"
I20260812 08:03:19.553339 19841 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task3yToRy/test-tmp/prometheus_sd-itest.0.MultiMasterPrometheusSDITest.EmptyArrayBehaviorOnFollower.1786521770778185-19841-0/minicluster-data/master-2-root
metadata directory: /tmp/dist-test-task3yToRy/test-tmp/prometheus_sd-itest.0.MultiMasterPrometheusSDITest.EmptyArrayBehaviorOnFollower.1786521770778185-19841-0/minicluster-data/master-2-root
1 data directories: /tmp/dist-test-task3yToRy/test-tmp/prometheus_sd-itest.0.MultiMasterPrometheusSDITest.EmptyArrayBehaviorOnFollower.1786521770778185-19841-0/minicluster-data/master-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 08:03:19.557037 21171 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.96.124" port: 45949 } has no permanent_uuid. Determining permanent_uuid...
W20260812 08:03:19.558548 21171 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.19.96.124:45949: Network error: Client connection negotiation failed: client connection to 127.19.96.124:45949: connect: Connection refused (error 111)
I20260812 08:03:19.560813 19841 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:03:19.561232 19841 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:03:19.562152 21089 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.96.125" port: 41391 } attempt: 1
I20260812 08:03:19.567198 19841 rpc_server.cc:307] RPC server started. Bound to: 127.19.96.124:45949
I20260812 08:03:19.567226 21249 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.19.96.124:45949 every 8 connection(s)
I20260812 08:03:19.567294 21089 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.96.124" port: 45949 } has no permanent_uuid. Determining permanent_uuid...
I20260812 08:03:19.567823 19841 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260812 08:03:19.568403 21250 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 08:03:19.569674 21250 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.96.126" port: 40893 } has no permanent_uuid. Determining permanent_uuid...
I20260812 08:03:19.573351 21089 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 05617cbaf05f4626a0992dae988ce9f9: Bootstrap starting.
I20260812 08:03:19.574155 21250 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.96.125" port: 41391 } has no permanent_uuid. Determining permanent_uuid...
I20260812 08:03:19.574627 21089 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 05617cbaf05f4626a0992dae988ce9f9: Neither blocks nor log segments found. Creating new log.
I20260812 08:03:19.575941 21089 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 05617cbaf05f4626a0992dae988ce9f9: No bootstrap required, opened a new log
I20260812 08:03:19.576613 21089 raft_consensus.cc:359] T 00000000000000000000000000000000 P 05617cbaf05f4626a0992dae988ce9f9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05617cbaf05f4626a0992dae988ce9f9" member_type: VOTER last_known_addr { host: "127.19.96.126" port: 40893 } } peers { permanent_uuid: "3d462115eb1741df9deb19ba42ac53aa" member_type: VOTER last_known_addr { host: "127.19.96.125" port: 41391 } } peers { permanent_uuid: "6d5c9a7153884097989bf782fd594c07" member_type: VOTER last_known_addr { host: "127.19.96.124" port: 45949 } }
I20260812 08:03:19.576761 21089 raft_consensus.cc:385] T 00000000000000000000000000000000 P 05617cbaf05f4626a0992dae988ce9f9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 08:03:19.576797 21089 raft_consensus.cc:740] T 00000000000000000000000000000000 P 05617cbaf05f4626a0992dae988ce9f9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 05617cbaf05f4626a0992dae988ce9f9, State: Initialized, Role: FOLLOWER
I20260812 08:03:19.576996 21089 consensus_queue.cc:260] T 00000000000000000000000000000000 P 05617cbaf05f4626a0992dae988ce9f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05617cbaf05f4626a0992dae988ce9f9" member_type: VOTER last_known_addr { host: "127.19.96.126" port: 40893 } } peers { permanent_uuid: "3d462115eb1741df9deb19ba42ac53aa" member_type: VOTER last_known_addr { host: "127.19.96.125" port: 41391 } } peers { permanent_uuid: "6d5c9a7153884097989bf782fd594c07" member_type: VOTER last_known_addr { host: "127.19.96.124" port: 45949 } }
I20260812 08:03:19.577343 21255 sys_catalog.cc:455] T 00000000000000000000000000000000 P 05617cbaf05f4626a0992dae988ce9f9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05617cbaf05f4626a0992dae988ce9f9" member_type: VOTER last_known_addr { host: "127.19.96.126" port: 40893 } } peers { permanent_uuid: "3d462115eb1741df9deb19ba42ac53aa" member_type: VOTER last_known_addr { host: "127.19.96.125" port: 41391 } } peers { permanent_uuid: "6d5c9a7153884097989bf782fd594c07" member_type: VOTER last_known_addr { host: "127.19.96.124" port: 45949 } } }
I20260812 08:03:19.577443 21255 sys_catalog.cc:458] T 00000000000000000000000000000000 P 05617cbaf05f4626a0992dae988ce9f9 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 08:03:19.577591 21089 sys_catalog.cc:565] T 00000000000000000000000000000000 P 05617cbaf05f4626a0992dae988ce9f9 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 08:03:19.578735 21250 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.96.124" port: 45949 } has no permanent_uuid. Determining permanent_uuid...
I20260812 08:03:19.579459 19841 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 1
W20260812 08:03:19.580019 21269 catalog_manager.cc:1594] T 00000000000000000000000000000000 P 05617cbaf05f4626a0992dae988ce9f9: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 08:03:19.580080 21269 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 08:03:19.584088 21250 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6d5c9a7153884097989bf782fd594c07: Bootstrap starting.
I20260812 08:03:19.585201 21250 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6d5c9a7153884097989bf782fd594c07: Neither blocks nor log segments found. Creating new log.
I20260812 08:03:19.586424 21250 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6d5c9a7153884097989bf782fd594c07: No bootstrap required, opened a new log
I20260812 08:03:19.586987 21250 raft_consensus.cc:359] T 00000000000000000000000000000000 P 6d5c9a7153884097989bf782fd594c07 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05617cbaf05f4626a0992dae988ce9f9" member_type: VOTER last_known_addr { host: "127.19.96.126" port: 40893 } } peers { permanent_uuid: "3d462115eb1741df9deb19ba42ac53aa" member_type: VOTER last_known_addr { host: "127.19.96.125" port: 41391 } } peers { permanent_uuid: "6d5c9a7153884097989bf782fd594c07" member_type: VOTER last_known_addr { host: "127.19.96.124" port: 45949 } }
I20260812 08:03:19.587091 21250 raft_consensus.cc:385] T 00000000000000000000000000000000 P 6d5c9a7153884097989bf782fd594c07 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 08:03:19.587138 21250 raft_consensus.cc:740] T 00000000000000000000000000000000 P 6d5c9a7153884097989bf782fd594c07 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6d5c9a7153884097989bf782fd594c07, State: Initialized, Role: FOLLOWER
I20260812 08:03:19.587319 21250 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6d5c9a7153884097989bf782fd594c07 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05617cbaf05f4626a0992dae988ce9f9" member_type: VOTER last_known_addr { host: "127.19.96.126" port: 40893 } } peers { permanent_uuid: "3d462115eb1741df9deb19ba42ac53aa" member_type: VOTER last_known_addr { host: "127.19.96.125" port: 41391 } } peers { permanent_uuid: "6d5c9a7153884097989bf782fd594c07" member_type: VOTER last_known_addr { host: "127.19.96.124" port: 45949 } }
I20260812 08:03:19.587689 21271 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6d5c9a7153884097989bf782fd594c07 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05617cbaf05f4626a0992dae988ce9f9" member_type: VOTER last_known_addr { host: "127.19.96.126" port: 40893 } } peers { permanent_uuid: "3d462115eb1741df9deb19ba42ac53aa" member_type: VOTER last_known_addr { host: "127.19.96.125" port: 41391 } } peers { permanent_uuid: "6d5c9a7153884097989bf782fd594c07" member_type: VOTER last_known_addr { host: "127.19.96.124" port: 45949 } } }
I20260812 08:03:19.587780 21271 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6d5c9a7153884097989bf782fd594c07 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 08:03:19.587904 21250 sys_catalog.cc:565] T 00000000000000000000000000000000 P 6d5c9a7153884097989bf782fd594c07 [sys.catalog]: configured and running, proceeding with master startup.
W20260812 08:03:19.590159 21283 catalog_manager.cc:1594] T 00000000000000000000000000000000 P 6d5c9a7153884097989bf782fd594c07: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 08:03:19.590232 21283 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260812 08:03:19.614749 21171 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.96.124" port: 45949 } attempt: 1
I20260812 08:03:19.620757 21171 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3d462115eb1741df9deb19ba42ac53aa: Bootstrap starting.
I20260812 08:03:19.621886 21171 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3d462115eb1741df9deb19ba42ac53aa: Neither blocks nor log segments found. Creating new log.
I20260812 08:03:19.623162 21171 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3d462115eb1741df9deb19ba42ac53aa: No bootstrap required, opened a new log
I20260812 08:03:19.623672 21171 raft_consensus.cc:359] T 00000000000000000000000000000000 P 3d462115eb1741df9deb19ba42ac53aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05617cbaf05f4626a0992dae988ce9f9" member_type: VOTER last_known_addr { host: "127.19.96.126" port: 40893 } } peers { permanent_uuid: "3d462115eb1741df9deb19ba42ac53aa" member_type: VOTER last_known_addr { host: "127.19.96.125" port: 41391 } } peers { permanent_uuid: "6d5c9a7153884097989bf782fd594c07" member_type: VOTER last_known_addr { host: "127.19.96.124" port: 45949 } }
I20260812 08:03:19.623776 21171 raft_consensus.cc:385] T 00000000000000000000000000000000 P 3d462115eb1741df9deb19ba42ac53aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 08:03:19.623799 21171 raft_consensus.cc:740] T 00000000000000000000000000000000 P 3d462115eb1741df9deb19ba42ac53aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d462115eb1741df9deb19ba42ac53aa, State: Initialized, Role: FOLLOWER
I20260812 08:03:19.624020 21171 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3d462115eb1741df9deb19ba42ac53aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05617cbaf05f4626a0992dae988ce9f9" member_type: VOTER last_known_addr { host: "127.19.96.126" port: 40893 } } peers { permanent_uuid: "3d462115eb1741df9deb19ba42ac53aa" member_type: VOTER last_known_addr { host: "127.19.96.125" port: 41391 } } peers { permanent_uuid: "6d5c9a7153884097989bf782fd594c07" member_type: VOTER last_known_addr { host: "127.19.96.124" port: 45949 } }
I20260812 08:03:19.624425 21285 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3d462115eb1741df9deb19ba42ac53aa [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05617cbaf05f4626a0992dae988ce9f9" member_type: VOTER last_known_addr { host: "127.19.96.126" port: 40893 } } peers { permanent_uuid: "3d462115eb1741df9deb19ba42ac53aa" member_type: VOTER last_known_addr { host: "127.19.96.125" port: 41391 } } peers { permanent_uuid: "6d5c9a7153884097989bf782fd594c07" member_type: VOTER last_known_addr { host: "127.19.96.124" port: 45949 } } }
I20260812 08:03:19.624540 21285 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3d462115eb1741df9deb19ba42ac53aa [sys.catalog]: This master's current role is: FOLLOWER
I20260812 08:03:19.624541 21171 sys_catalog.cc:565] T 00000000000000000000000000000000 P 3d462115eb1741df9deb19ba42ac53aa [sys.catalog]: configured and running, proceeding with master startup.
I20260812 08:03:19.626411 19841 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 2
I20260812 08:03:19.626961 19841 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 08:03:19.626997 21297 catalog_manager.cc:1594] T 00000000000000000000000000000000 P 3d462115eb1741df9deb19ba42ac53aa: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260812 08:03:19.627126 21297 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260812 08:03:19.629344 21303 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:03:19.629338 21298 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:03:19.629402 21299 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:03:19.629626 19841 server_base.cc:1061] running on GCE node
I20260812 08:03:19.629837 19841 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260812 08:03:19.629900 19841 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260812 08:03:19.629936 19841 hybrid_clock.cc:648] HybridClock initialized: now 1786521799629935 us; error 0 us; skew 500 ppm
I20260812 08:03:19.630955 19841 webserver.cc:533] Webserver started at http://127.19.96.65:36461/ using document root <none> and password file <none>
I20260812 08:03:19.631151 19841 fs_manager.cc:362] Metadata directory not provided
I20260812 08:03:19.631225 19841 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 08:03:19.631306 19841 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 08:03:19.631724 19841 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task3yToRy/test-tmp/prometheus_sd-itest.0.MultiMasterPrometheusSDITest.EmptyArrayBehaviorOnFollower.1786521770778185-19841-0/minicluster-data/ts-0-root/instance:
uuid: "14e57ba07d0149aa8acb81a1c5680a8f"
format_stamp: "Formatted at 2026-08-12 08:03:19 on dist-test-slave-8w9v"
I20260812 08:03:19.633404 19841 fs_manager.cc:696] Time spent creating directory manager: real 0.001s	user 0.000s	sys 0.002s
I20260812 08:03:19.634445 21308 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:03:19.634761 19841 fs_manager.cc:730] Time spent opening block manager: real 0.001s	user 0.000s	sys 0.001s
I20260812 08:03:19.634848 19841 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task3yToRy/test-tmp/prometheus_sd-itest.0.MultiMasterPrometheusSDITest.EmptyArrayBehaviorOnFollower.1786521770778185-19841-0/minicluster-data/ts-0-root
uuid: "14e57ba07d0149aa8acb81a1c5680a8f"
format_stamp: "Formatted at 2026-08-12 08:03:19 on dist-test-slave-8w9v"
I20260812 08:03:19.634948 19841 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task3yToRy/test-tmp/prometheus_sd-itest.0.MultiMasterPrometheusSDITest.EmptyArrayBehaviorOnFollower.1786521770778185-19841-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-task3yToRy/test-tmp/prometheus_sd-itest.0.MultiMasterPrometheusSDITest.EmptyArrayBehaviorOnFollower.1786521770778185-19841-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-task3yToRy/test-tmp/prometheus_sd-itest.0.MultiMasterPrometheusSDITest.EmptyArrayBehaviorOnFollower.1786521770778185-19841-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 08:03:19.650812 19841 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:03:19.651328 19841 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:03:19.651688 19841 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260812 08:03:19.652307 19841 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 08:03:19.652372 19841 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:03:19.652437 19841 ts_tablet_manager.cc:616] Registered 0 tablets
I20260812 08:03:19.652488 19841 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:03:19.657877 19841 rpc_server.cc:307] RPC server started. Bound to: 127.19.96.65:45685
I20260812 08:03:19.658033 21378 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.19.96.65:45685 every 8 connection(s)
I20260812 08:03:19.663700 21379 heartbeater.cc:344] Connected to a master server at 127.19.96.124:45949
I20260812 08:03:19.663841 21379 heartbeater.cc:461] Registering TS with master...
I20260812 08:03:19.663916 21380 heartbeater.cc:344] Connected to a master server at 127.19.96.126:40893
I20260812 08:03:19.663976 21380 heartbeater.cc:461] Registering TS with master...
I20260812 08:03:19.664100 21379 heartbeater.cc:507] Master 127.19.96.124:45949 requested a full tablet report, sending...
I20260812 08:03:19.664151 21380 heartbeater.cc:507] Master 127.19.96.126:40893 requested a full tablet report, sending...
I20260812 08:03:19.664274 21382 heartbeater.cc:344] Connected to a master server at 127.19.96.125:41391
I20260812 08:03:19.664366 21382 heartbeater.cc:461] Registering TS with master...
I20260812 08:03:19.664572 21382 heartbeater.cc:507] Master 127.19.96.125:41391 requested a full tablet report, sending...
I20260812 08:03:19.664877 21046 ts_manager.cc:194] Registered new tserver with Master: 14e57ba07d0149aa8acb81a1c5680a8f (127.19.96.65:45685)
I20260812 08:03:19.664889 21206 ts_manager.cc:194] Registered new tserver with Master: 14e57ba07d0149aa8acb81a1c5680a8f (127.19.96.65:45685)
I20260812 08:03:19.665266 21128 ts_manager.cc:194] Registered new tserver with Master: 14e57ba07d0149aa8acb81a1c5680a8f (127.19.96.65:45685)
I20260812 08:03:19.665742 19841 internal_mini_cluster.cc:371] 1 TS(s) registered with all masters after 0.007233141s
I20260812 08:03:19.666296 19841 internal_mini_cluster.cc:371] 1 TS(s) registered with all masters after 3.9149e-05s
time=2026-08-12T08:03:19.738Z level=INFO source=main.go:1678 msg="updated GOGC" old=100 new=75
time=2026-08-12T08:03:19.739Z level=INFO source=main.go:744 msg="Leaving GOMAXPROCS=4: CPU quota undefined" component=automaxprocs
time=2026-08-12T08:03:19.740Z level=INFO source=memlimit.go:198 msg="GOMEMLIMIT is updated" component=automemlimit package=github.com/KimMachineGun/automemlimit/memlimit GOMEMLIMIT=14142552883 previous=9223372036854775807
time=2026-08-12T08:03:19.740Z level=INFO source=main.go:851 msg="Starting Prometheus Server" mode=server version="(version=3.11.2, branch=HEAD, revision=f0f0fdd679dcd6df320b0558b20919f7cd44c407)"
time=2026-08-12T08:03:19.740Z level=INFO source=main.go:856 msg="operational information" build_context="(go=go1.26.2, platform=linux/amd64, user=root@d2684485c347, date=20260413-11:53:40, tags=netgo,builtinassets)" host_details="(Linux 5.4.0-1157-gcp #166~18.04.1-Ubuntu SMP Wed Jan 21 01:57:46 UTC 2026 x86_64 dist-test-slave-8w9v (none))" fd_limits="(soft=1048576, hard=1048576)" vm_limits="(soft=unlimited, hard=unlimited)"
W20260812 08:03:19.745565 21035 tablet.cc:2406] T 00000000000000000000000000000000 P 05617cbaf05f4626a0992dae988ce9f9: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260812 08:03:19.779644 21255 raft_consensus.cc:493] T 00000000000000000000000000000000 P 05617cbaf05f4626a0992dae988ce9f9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260812 08:03:19.779872 21255 raft_consensus.cc:515] T 00000000000000000000000000000000 P 05617cbaf05f4626a0992dae988ce9f9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05617cbaf05f4626a0992dae988ce9f9" member_type: VOTER last_known_addr { host: "127.19.96.126" port: 40893 } } peers { permanent_uuid: "3d462115eb1741df9deb19ba42ac53aa" member_type: VOTER last_known_addr { host: "127.19.96.125" port: 41391 } } peers { permanent_uuid: "6d5c9a7153884097989bf782fd594c07" member_type: VOTER last_known_addr { host: "127.19.96.124" port: 45949 } }
I20260812 08:03:19.780714 21255 leader_election.cc:290] T 00000000000000000000000000000000 P 05617cbaf05f4626a0992dae988ce9f9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3d462115eb1741df9deb19ba42ac53aa (127.19.96.125:41391), 6d5c9a7153884097989bf782fd594c07 (127.19.96.124:45949)
I20260812 08:03:19.781556 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "05617cbaf05f4626a0992dae988ce9f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d462115eb1741df9deb19ba42ac53aa" is_pre_election: true
I20260812 08:03:19.781708 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "05617cbaf05f4626a0992dae988ce9f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d5c9a7153884097989bf782fd594c07" is_pre_election: true
I20260812 08:03:19.781769 21143 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 3d462115eb1741df9deb19ba42ac53aa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05617cbaf05f4626a0992dae988ce9f9 in term 0.
I20260812 08:03:19.781906 21216 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 6d5c9a7153884097989bf782fd594c07 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05617cbaf05f4626a0992dae988ce9f9 in term 0.
I20260812 08:03:19.782250 21033 leader_election.cc:304] T 00000000000000000000000000000000 P 05617cbaf05f4626a0992dae988ce9f9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05617cbaf05f4626a0992dae988ce9f9, 3d462115eb1741df9deb19ba42ac53aa; no voters: 
I20260812 08:03:19.782490 21255 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 05617cbaf05f4626a0992dae988ce9f9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260812 08:03:19.782581 21255 raft_consensus.cc:493] T 00000000000000000000000000000000 P 05617cbaf05f4626a0992dae988ce9f9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260812 08:03:19.782648 21255 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 05617cbaf05f4626a0992dae988ce9f9 [term 0 FOLLOWER]: Advancing to term 1
I20260812 08:03:19.784679 21255 raft_consensus.cc:515] T 00000000000000000000000000000000 P 05617cbaf05f4626a0992dae988ce9f9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05617cbaf05f4626a0992dae988ce9f9" member_type: VOTER last_known_addr { host: "127.19.96.126" port: 40893 } } peers { permanent_uuid: "3d462115eb1741df9deb19ba42ac53aa" member_type: VOTER last_known_addr { host: "127.19.96.125" port: 41391 } } peers { permanent_uuid: "6d5c9a7153884097989bf782fd594c07" member_type: VOTER last_known_addr { host: "127.19.96.124" port: 45949 } }
I20260812 08:03:19.785830 21143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "05617cbaf05f4626a0992dae988ce9f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d462115eb1741df9deb19ba42ac53aa"
I20260812 08:03:19.786020 21143 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 3d462115eb1741df9deb19ba42ac53aa [term 0 FOLLOWER]: Advancing to term 1
I20260812 08:03:19.786218 21255 leader_election.cc:290] T 00000000000000000000000000000000 P 05617cbaf05f4626a0992dae988ce9f9 [CANDIDATE]: Term 1 election: Requested vote from peers 3d462115eb1741df9deb19ba42ac53aa (127.19.96.125:41391), 6d5c9a7153884097989bf782fd594c07 (127.19.96.124:45949)
I20260812 08:03:19.786759 21216 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "05617cbaf05f4626a0992dae988ce9f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d5c9a7153884097989bf782fd594c07"
I20260812 08:03:19.786885 21216 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 6d5c9a7153884097989bf782fd594c07 [term 0 FOLLOWER]: Advancing to term 1
I20260812 08:03:19.787829 21143 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 3d462115eb1741df9deb19ba42ac53aa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05617cbaf05f4626a0992dae988ce9f9 in term 1.
I20260812 08:03:19.787990 21216 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 6d5c9a7153884097989bf782fd594c07 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05617cbaf05f4626a0992dae988ce9f9 in term 1.
I20260812 08:03:19.788403 21033 leader_election.cc:304] T 00000000000000000000000000000000 P 05617cbaf05f4626a0992dae988ce9f9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05617cbaf05f4626a0992dae988ce9f9, 3d462115eb1741df9deb19ba42ac53aa; no voters: 
I20260812 08:03:19.788707 21255 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 05617cbaf05f4626a0992dae988ce9f9 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 08:03:19.789022 21255 raft_consensus.cc:697] T 00000000000000000000000000000000 P 05617cbaf05f4626a0992dae988ce9f9 [term 1 LEADER]: Becoming Leader. State: Replica: 05617cbaf05f4626a0992dae988ce9f9, State: Running, Role: LEADER
I20260812 08:03:19.789335 21255 consensus_queue.cc:237] T 00000000000000000000000000000000 P 05617cbaf05f4626a0992dae988ce9f9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05617cbaf05f4626a0992dae988ce9f9" member_type: VOTER last_known_addr { host: "127.19.96.126" port: 40893 } } peers { permanent_uuid: "3d462115eb1741df9deb19ba42ac53aa" member_type: VOTER last_known_addr { host: "127.19.96.125" port: 41391 } } peers { permanent_uuid: "6d5c9a7153884097989bf782fd594c07" member_type: VOTER last_known_addr { host: "127.19.96.124" port: 45949 } }
I20260812 08:03:19.790887 21400 sys_catalog.cc:455] T 00000000000000000000000000000000 P 05617cbaf05f4626a0992dae988ce9f9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 05617cbaf05f4626a0992dae988ce9f9. Latest consensus state: current_term: 1 leader_uuid: "05617cbaf05f4626a0992dae988ce9f9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05617cbaf05f4626a0992dae988ce9f9" member_type: VOTER last_known_addr { host: "127.19.96.126" port: 40893 } } peers { permanent_uuid: "3d462115eb1741df9deb19ba42ac53aa" member_type: VOTER last_known_addr { host: "127.19.96.125" port: 41391 } } peers { permanent_uuid: "6d5c9a7153884097989bf782fd594c07" member_type: VOTER last_known_addr { host: "127.19.96.124" port: 45949 } } }
I20260812 08:03:19.791136 21400 sys_catalog.cc:458] T 00000000000000000000000000000000 P 05617cbaf05f4626a0992dae988ce9f9 [sys.catalog]: This master's current role is: LEADER
I20260812 08:03:19.791661 21404 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260812 08:03:19.792618 21404 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260812 08:03:19.795228 21143 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 3d462115eb1741df9deb19ba42ac53aa [term 1 FOLLOWER]: Refusing update from remote peer 05617cbaf05f4626a0992dae988ce9f9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 08:03:19.795254 21216 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 6d5c9a7153884097989bf782fd594c07 [term 1 FOLLOWER]: Refusing update from remote peer 05617cbaf05f4626a0992dae988ce9f9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260812 08:03:19.795665 21400 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 05617cbaf05f4626a0992dae988ce9f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d462115eb1741df9deb19ba42ac53aa" member_type: VOTER last_known_addr { host: "127.19.96.125" port: 41391 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 08:03:19.796030 21400 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 05617cbaf05f4626a0992dae988ce9f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6d5c9a7153884097989bf782fd594c07" member_type: VOTER last_known_addr { host: "127.19.96.124" port: 45949 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260812 08:03:19.797500 21285 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3d462115eb1741df9deb19ba42ac53aa [sys.catalog]: SysCatalogTable state changed. Reason: New leader 05617cbaf05f4626a0992dae988ce9f9. Latest consensus state: current_term: 1 leader_uuid: "05617cbaf05f4626a0992dae988ce9f9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05617cbaf05f4626a0992dae988ce9f9" member_type: VOTER last_known_addr { host: "127.19.96.126" port: 40893 } } peers { permanent_uuid: "3d462115eb1741df9deb19ba42ac53aa" member_type: VOTER last_known_addr { host: "127.19.96.125" port: 41391 } } peers { permanent_uuid: "6d5c9a7153884097989bf782fd594c07" member_type: VOTER last_known_addr { host: "127.19.96.124" port: 45949 } } }
I20260812 08:03:19.797614 21285 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3d462115eb1741df9deb19ba42ac53aa [sys.catalog]: This master's current role is: FOLLOWER
I20260812 08:03:19.797963 21271 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6d5c9a7153884097989bf782fd594c07 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 05617cbaf05f4626a0992dae988ce9f9. Latest consensus state: current_term: 1 leader_uuid: "05617cbaf05f4626a0992dae988ce9f9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05617cbaf05f4626a0992dae988ce9f9" member_type: VOTER last_known_addr { host: "127.19.96.126" port: 40893 } } peers { permanent_uuid: "3d462115eb1741df9deb19ba42ac53aa" member_type: VOTER last_known_addr { host: "127.19.96.125" port: 41391 } } peers { permanent_uuid: "6d5c9a7153884097989bf782fd594c07" member_type: VOTER last_known_addr { host: "127.19.96.124" port: 45949 } } }
I20260812 08:03:19.798105 21271 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6d5c9a7153884097989bf782fd594c07 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 08:03:19.799217 21400 sys_catalog.cc:455] T 00000000000000000000000000000000 P 05617cbaf05f4626a0992dae988ce9f9 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "05617cbaf05f4626a0992dae988ce9f9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05617cbaf05f4626a0992dae988ce9f9" member_type: VOTER last_known_addr { host: "127.19.96.126" port: 40893 } } peers { permanent_uuid: "3d462115eb1741df9deb19ba42ac53aa" member_type: VOTER last_known_addr { host: "127.19.96.125" port: 41391 } } peers { permanent_uuid: "6d5c9a7153884097989bf782fd594c07" member_type: VOTER last_known_addr { host: "127.19.96.124" port: 45949 } } }
I20260812 08:03:19.799319 21400 sys_catalog.cc:458] T 00000000000000000000000000000000 P 05617cbaf05f4626a0992dae988ce9f9 [sys.catalog]: This master's current role is: LEADER
I20260812 08:03:19.800261 21404 catalog_manager.cc:1383] Generated new cluster ID: 0d6bd1917bcb40d2aa2b19d8119763a6
I20260812 08:03:19.800318 21404 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260812 08:03:19.800623 21271 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6d5c9a7153884097989bf782fd594c07 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "05617cbaf05f4626a0992dae988ce9f9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05617cbaf05f4626a0992dae988ce9f9" member_type: VOTER last_known_addr { host: "127.19.96.126" port: 40893 } } peers { permanent_uuid: "3d462115eb1741df9deb19ba42ac53aa" member_type: VOTER last_known_addr { host: "127.19.96.125" port: 41391 } } peers { permanent_uuid: "6d5c9a7153884097989bf782fd594c07" member_type: VOTER last_known_addr { host: "127.19.96.124" port: 45949 } } }
I20260812 08:03:19.800760 21271 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6d5c9a7153884097989bf782fd594c07 [sys.catalog]: This master's current role is: FOLLOWER
I20260812 08:03:19.803390 21255 sys_catalog.cc:455] T 00000000000000000000000000000000 P 05617cbaf05f4626a0992dae988ce9f9 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "05617cbaf05f4626a0992dae988ce9f9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05617cbaf05f4626a0992dae988ce9f9" member_type: VOTER last_known_addr { host: "127.19.96.126" port: 40893 } } peers { permanent_uuid: "3d462115eb1741df9deb19ba42ac53aa" member_type: VOTER last_known_addr { host: "127.19.96.125" port: 41391 } } peers { permanent_uuid: "6d5c9a7153884097989bf782fd594c07" member_type: VOTER last_known_addr { host: "127.19.96.124" port: 45949 } } }
I20260812 08:03:19.803573 21255 sys_catalog.cc:458] T 00000000000000000000000000000000 P 05617cbaf05f4626a0992dae988ce9f9 [sys.catalog]: This master's current role is: LEADER
I20260812 08:03:19.804944 21285 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3d462115eb1741df9deb19ba42ac53aa [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "05617cbaf05f4626a0992dae988ce9f9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05617cbaf05f4626a0992dae988ce9f9" member_type: VOTER last_known_addr { host: "127.19.96.126" port: 40893 } } peers { permanent_uuid: "3d462115eb1741df9deb19ba42ac53aa" member_type: VOTER last_known_addr { host: "127.19.96.125" port: 41391 } } peers { permanent_uuid: "6d5c9a7153884097989bf782fd594c07" member_type: VOTER last_known_addr { host: "127.19.96.124" port: 45949 } } }
I20260812 08:03:19.805094 21285 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3d462115eb1741df9deb19ba42ac53aa [sys.catalog]: This master's current role is: FOLLOWER
I20260812 08:03:19.873739 21404 catalog_manager.cc:1406] Generated new certificate authority record
I20260812 08:03:19.874662 21404 catalog_manager.cc:1540] Loading token signing keys...
I20260812 08:03:19.895723 21404 catalog_manager.cc:6092] T 00000000000000000000000000000000 P 05617cbaf05f4626a0992dae988ce9f9: Generated new TSK 0
I20260812 08:03:19.895962 21404 catalog_manager.cc:1550] Initializing in-progress tserver states...
time=2026-08-12T08:03:20.002Z level=INFO source=web.go:710 msg="Start listening for connections" component=web address=127.0.0.1:0
time=2026-08-12T08:03:20.003Z level=INFO source=main.go:1410 msg="Starting TSDB ..."
time=2026-08-12T08:03:20.009Z level=INFO source=tls_config.go:354 msg="Listening on" component=web address=127.0.0.1:44851
time=2026-08-12T08:03:20.009Z level=INFO source=tls_config.go:357 msg="TLS is disabled." component=web http2=false address=127.0.0.1:44851
time=2026-08-12T08:03:20.012Z level=INFO source=head.go:698 msg="Replaying on-disk memory mappable chunks if any" component=tsdb
time=2026-08-12T08:03:20.012Z level=INFO source=head.go:784 msg="On-disk memory mappable chunks replay completed" component=tsdb duration=1.971µs
time=2026-08-12T08:03:20.012Z level=INFO source=head.go:792 msg="Replaying WAL, this may take a while" component=tsdb
time=2026-08-12T08:03:20.013Z level=INFO source=head.go:865 msg="WAL segment loaded" component=tsdb segment=0 maxSegment=0 duration=335.505µs
time=2026-08-12T08:03:20.013Z level=INFO source=head.go:902 msg="WAL replay completed" component=tsdb checkpoint_replay_duration=41.448µs wal_replay_duration=377.069µs wbl_replay_duration=168ns chunk_snapshot_load_duration=0s mmap_chunk_replay_duration=1.971µs total_replay_duration=448.082µs
time=2026-08-12T08:03:20.015Z level=INFO source=main.go:1431 msg="filesystem information" fs_type=OVERLAYFS_SUPER_MAGIC
time=2026-08-12T08:03:20.015Z level=INFO source=main.go:1434 msg="TSDB started"
time=2026-08-12T08:03:20.015Z level=INFO source=main.go:1632 msg="Loading configuration file" filename=/tmp/dist-test-task3yToRy/test-tmp/prometheus_sd-itest.0.MultiMasterPrometheusSDITest.EmptyArrayBehaviorOnFollower.1786521770778185-19841-0/prometheus.yml
time=2026-08-12T08:03:20.015Z level=INFO source=main.go:1048 msg="TSDB retention updated" duration=15d size=0B percentage=0
time=2026-08-12T08:03:20.017Z level=INFO source=main.go:1671 msg="Completed loading of configuration file" db_storage=44.707µs remote_storage=2.276µs web_handler=1.059µs query_engine=1.269µs scrape=1.383212ms scrape_sd=303.367µs notify=2.797µs notify_sd=1.978µs rules=3.104µs tracing=7.099µs filename=/tmp/dist-test-task3yToRy/test-tmp/prometheus_sd-itest.0.MultiMasterPrometheusSDITest.EmptyArrayBehaviorOnFollower.1786521770778185-19841-0/prometheus.yml totalDuration=2.131681ms
time=2026-08-12T08:03:20.017Z level=INFO source=main.go:1395 msg="Server is ready to receive web requests."
time=2026-08-12T08:03:20.017Z level=INFO source=manager.go:209 msg="Starting rule manager..." component="rule manager"
I20260812 08:03:20.591821 21283 catalog_manager.cc:1295] Loaded cluster ID: 0d6bd1917bcb40d2aa2b19d8119763a6
I20260812 08:03:20.592000 21283 catalog_manager.cc:1588] T 00000000000000000000000000000000 P 6d5c9a7153884097989bf782fd594c07: loading cluster ID for follower catalog manager: success
I20260812 08:03:20.594762 21283 catalog_manager.cc:1610] T 00000000000000000000000000000000 P 6d5c9a7153884097989bf782fd594c07: acquiring CA information for follower catalog manager: success
I20260812 08:03:20.596114 21283 catalog_manager.cc:1638] T 00000000000000000000000000000000 P 6d5c9a7153884097989bf782fd594c07: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260812 08:03:20.628166 21297 catalog_manager.cc:1295] Loaded cluster ID: 0d6bd1917bcb40d2aa2b19d8119763a6
I20260812 08:03:20.628295 21297 catalog_manager.cc:1588] T 00000000000000000000000000000000 P 3d462115eb1741df9deb19ba42ac53aa: loading cluster ID for follower catalog manager: success
I20260812 08:03:20.629599 21297 catalog_manager.cc:1610] T 00000000000000000000000000000000 P 3d462115eb1741df9deb19ba42ac53aa: acquiring CA information for follower catalog manager: success
I20260812 08:03:20.630218 21297 catalog_manager.cc:1638] T 00000000000000000000000000000000 P 3d462115eb1741df9deb19ba42ac53aa: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260812 08:03:20.667910 21046 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:60454
I20260812 08:03:21.669386 21380 heartbeater.cc:499] Master 127.19.96.126:40893 was elected leader, sending a full tablet report...
I20260812 08:03:27.583151 21269 catalog_manager.cc:1638] T 00000000000000000000000000000000 P 05617cbaf05f4626a0992dae988ce9f9: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260812 08:03:33.687041 21380 heartbeater.cc:499] Master 127.19.96.126:40893 was elected leader, sending a full tablet report...
time=2026-08-12T08:03:36.845Z level=WARN source=main.go:1191 msg="Received an OS signal, exiting gracefully..." signal=terminated
time=2026-08-12T08:03:36.845Z level=INFO source=main.go:1216 msg="Stopping scrape discovery manager..."
time=2026-08-12T08:03:36.845Z level=INFO source=main.go:1230 msg="Stopping notify discovery manager..."
time=2026-08-12T08:03:36.845Z level=INFO source=main.go:1245 msg="Stopping rule manager manager..."
time=2026-08-12T08:03:36.845Z level=INFO source=manager.go:223 msg="Stopping rule manager..." component="rule manager"
time=2026-08-12T08:03:36.845Z level=INFO source=manager.go:239 msg="Rule manager stopped" component="rule manager"
time=2026-08-12T08:03:36.845Z level=INFO source=main.go:1269 msg="Stopping scrape manager..."
time=2026-08-12T08:03:36.848Z level=INFO source=main.go:1241 msg="Rule manager stopped"
time=2026-08-12T08:03:36.848Z level=INFO source=main.go:1261 msg="Scrape manager stopped"
time=2026-08-12T08:03:36.848Z level=INFO source=main.go:1212 msg="Scrape discovery manager stopped"
time=2026-08-12T08:03:36.848Z level=INFO source=main.go:1284 msg="Stopping tracing manager..."
time=2026-08-12T08:03:36.848Z level=INFO source=main.go:1369 msg="Stopping reloaders..."
time=2026-08-12T08:03:36.848Z level=INFO source=main.go:1226 msg="Notify discovery manager stopped"
time=2026-08-12T08:03:36.848Z level=INFO source=main.go:1458 msg="Stopping storage..."
time=2026-08-12T08:03:36.848Z level=INFO source=main.go:1280 msg="Tracing manager stopped"
time=2026-08-12T08:03:36.848Z level=INFO source=main.go:1361 msg="Reloaders stopped"
time=2026-08-12T08:03:36.853Z level=INFO source=main.go:1536 msg="Stopping web handler..."
time=2026-08-12T08:03:36.854Z level=INFO source=main.go:1559 msg="Stopping notifier manager..."
time=2026-08-12T08:03:36.854Z level=INFO source=manager.go:321 msg="Stopping notification manager..." component=notifier
time=2026-08-12T08:03:36.854Z level=INFO source=main.go:1454 msg="TSDB stopped"
time=2026-08-12T08:03:36.854Z level=INFO source=main.go:1555 msg="Notifier manager stopped"
time=2026-08-12T08:03:36.854Z level=INFO source=main.go:1532 msg="Web handler stopped"
time=2026-08-12T08:03:36.854Z level=INFO source=main.go:1570 msg="See you next time!"
I20260812 08:03:36.866291 19841 tablet_server.cc:179] TabletServer@127.19.96.65:0 shutting down...
I20260812 08:03:36.870671 19841 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260812 08:03:36.883214 19841 tablet_server.cc:196] TabletServer@127.19.96.65:0 shutdown complete.
I20260812 08:03:36.885452 19841 master.cc:562] Master@127.19.96.126:40893 shutting down...
I20260812 08:03:36.889381 19841 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 05617cbaf05f4626a0992dae988ce9f9 [term 1 LEADER]: Raft consensus shutting down.
I20260812 08:03:36.889698 19841 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 05617cbaf05f4626a0992dae988ce9f9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 08:03:36.889792 19841 tablet_replica.cc:333] T 00000000000000000000000000000000 P 05617cbaf05f4626a0992dae988ce9f9: stopping tablet replica
I20260812 08:03:36.902736 19841 master.cc:584] Master@127.19.96.126:40893 shutdown complete.
I20260812 08:03:36.905897 19841 master.cc:562] Master@127.19.96.125:41391 shutting down...
I20260812 08:03:36.910013 19841 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 3d462115eb1741df9deb19ba42ac53aa [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 08:03:36.910178 19841 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 3d462115eb1741df9deb19ba42ac53aa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 08:03:36.910228 19841 tablet_replica.cc:333] T 00000000000000000000000000000000 P 3d462115eb1741df9deb19ba42ac53aa: stopping tablet replica
I20260812 08:03:36.923084 19841 master.cc:584] Master@127.19.96.125:41391 shutdown complete.
I20260812 08:03:36.925884 19841 master.cc:562] Master@127.19.96.124:45949 shutting down...
I20260812 08:03:36.930341 19841 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 6d5c9a7153884097989bf782fd594c07 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260812 08:03:36.930620 19841 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 6d5c9a7153884097989bf782fd594c07 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 08:03:36.930706 19841 tablet_replica.cc:333] T 00000000000000000000000000000000 P 6d5c9a7153884097989bf782fd594c07: stopping tablet replica
I20260812 08:03:36.943756 19841 master.cc:584] Master@127.19.96.124:45949 shutdown complete.
[       OK ] MultiMasterPrometheusSDITest.EmptyArrayBehaviorOnFollower (17482 ms)
[----------] 2 tests from MultiMasterPrometheusSDITest (24947 ms total)

[----------] 1 test from LocationLabelSDITest
[ RUN      ] LocationLabelSDITest.TserverLocationLabel
I20260812 08:03:36.950609 19841 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.19.96.126:44203
I20260812 08:03:36.950989 19841 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 08:03:36.953297 21434 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:03:36.953383 21432 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:03:36.953305 21431 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:03:36.953472 19841 server_base.cc:1061] running on GCE node
I20260812 08:03:36.953706 19841 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260812 08:03:36.953768 19841 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260812 08:03:36.953794 19841 hybrid_clock.cc:648] HybridClock initialized: now 1786521816953794 us; error 0 us; skew 500 ppm
I20260812 08:03:36.954715 19841 webserver.cc:533] Webserver started at http://127.19.96.126:34229/ using document root <none> and password file <none>
I20260812 08:03:36.954901 19841 fs_manager.cc:362] Metadata directory not provided
I20260812 08:03:36.954974 19841 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 08:03:36.955058 19841 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 08:03:36.955525 19841 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task3yToRy/test-tmp/prometheus_sd-itest.0.LocationLabelSDITest.TserverLocationLabel.1786521770778185-19841-0/minicluster-data/master-0-root/instance:
uuid: "44bff267e38e484880e2aefb7bc05807"
format_stamp: "Formatted at 2026-08-12 08:03:36 on dist-test-slave-8w9v"
I20260812 08:03:36.957185 19841 fs_manager.cc:696] Time spent creating directory manager: real 0.001s	user 0.002s	sys 0.000s
I20260812 08:03:36.958226 21439 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:03:36.958613 19841 fs_manager.cc:730] Time spent opening block manager: real 0.001s	user 0.001s	sys 0.000s
I20260812 08:03:36.958709 19841 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task3yToRy/test-tmp/prometheus_sd-itest.0.LocationLabelSDITest.TserverLocationLabel.1786521770778185-19841-0/minicluster-data/master-0-root
uuid: "44bff267e38e484880e2aefb7bc05807"
format_stamp: "Formatted at 2026-08-12 08:03:36 on dist-test-slave-8w9v"
I20260812 08:03:36.958798 19841 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task3yToRy/test-tmp/prometheus_sd-itest.0.LocationLabelSDITest.TserverLocationLabel.1786521770778185-19841-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-task3yToRy/test-tmp/prometheus_sd-itest.0.LocationLabelSDITest.TserverLocationLabel.1786521770778185-19841-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-task3yToRy/test-tmp/prometheus_sd-itest.0.LocationLabelSDITest.TserverLocationLabel.1786521770778185-19841-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 08:03:36.971599 19841 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:03:36.972064 19841 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:03:36.977219 19841 rpc_server.cc:307] RPC server started. Bound to: 127.19.96.126:44203
I20260812 08:03:36.979307 21497 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260812 08:03:36.980226 21496 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.19.96.126:44203 every 8 connection(s)
I20260812 08:03:36.981654 21497 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 44bff267e38e484880e2aefb7bc05807: Bootstrap starting.
I20260812 08:03:36.982543 21497 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 44bff267e38e484880e2aefb7bc05807: Neither blocks nor log segments found. Creating new log.
I20260812 08:03:36.983639 21497 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 44bff267e38e484880e2aefb7bc05807: No bootstrap required, opened a new log
I20260812 08:03:36.984100 21497 raft_consensus.cc:359] T 00000000000000000000000000000000 P 44bff267e38e484880e2aefb7bc05807 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44bff267e38e484880e2aefb7bc05807" member_type: VOTER }
I20260812 08:03:36.984256 21497 raft_consensus.cc:385] T 00000000000000000000000000000000 P 44bff267e38e484880e2aefb7bc05807 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260812 08:03:36.984309 21497 raft_consensus.cc:740] T 00000000000000000000000000000000 P 44bff267e38e484880e2aefb7bc05807 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 44bff267e38e484880e2aefb7bc05807, State: Initialized, Role: FOLLOWER
I20260812 08:03:36.984470 21497 consensus_queue.cc:260] T 00000000000000000000000000000000 P 44bff267e38e484880e2aefb7bc05807 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44bff267e38e484880e2aefb7bc05807" member_type: VOTER }
I20260812 08:03:36.984565 21497 raft_consensus.cc:399] T 00000000000000000000000000000000 P 44bff267e38e484880e2aefb7bc05807 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260812 08:03:36.984609 21497 raft_consensus.cc:493] T 00000000000000000000000000000000 P 44bff267e38e484880e2aefb7bc05807 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260812 08:03:36.984664 21497 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 44bff267e38e484880e2aefb7bc05807 [term 0 FOLLOWER]: Advancing to term 1
I20260812 08:03:36.985360 21497 raft_consensus.cc:515] T 00000000000000000000000000000000 P 44bff267e38e484880e2aefb7bc05807 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44bff267e38e484880e2aefb7bc05807" member_type: VOTER }
I20260812 08:03:36.985518 21497 leader_election.cc:304] T 00000000000000000000000000000000 P 44bff267e38e484880e2aefb7bc05807 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 44bff267e38e484880e2aefb7bc05807; no voters: 
I20260812 08:03:36.985736 21497 leader_election.cc:290] T 00000000000000000000000000000000 P 44bff267e38e484880e2aefb7bc05807 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260812 08:03:36.985921 21500 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 44bff267e38e484880e2aefb7bc05807 [term 1 FOLLOWER]: Leader election won for term 1
I20260812 08:03:36.986157 21500 raft_consensus.cc:697] T 00000000000000000000000000000000 P 44bff267e38e484880e2aefb7bc05807 [term 1 LEADER]: Becoming Leader. State: Replica: 44bff267e38e484880e2aefb7bc05807, State: Running, Role: LEADER
I20260812 08:03:36.986263 21497 sys_catalog.cc:565] T 00000000000000000000000000000000 P 44bff267e38e484880e2aefb7bc05807 [sys.catalog]: configured and running, proceeding with master startup.
I20260812 08:03:36.986351 21500 consensus_queue.cc:237] T 00000000000000000000000000000000 P 44bff267e38e484880e2aefb7bc05807 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44bff267e38e484880e2aefb7bc05807" member_type: VOTER }
I20260812 08:03:36.986833 21501 sys_catalog.cc:455] T 00000000000000000000000000000000 P 44bff267e38e484880e2aefb7bc05807 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "44bff267e38e484880e2aefb7bc05807" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44bff267e38e484880e2aefb7bc05807" member_type: VOTER } }
I20260812 08:03:36.986866 21502 sys_catalog.cc:455] T 00000000000000000000000000000000 P 44bff267e38e484880e2aefb7bc05807 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 44bff267e38e484880e2aefb7bc05807. Latest consensus state: current_term: 1 leader_uuid: "44bff267e38e484880e2aefb7bc05807" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44bff267e38e484880e2aefb7bc05807" member_type: VOTER } }
I20260812 08:03:36.986943 21501 sys_catalog.cc:458] T 00000000000000000000000000000000 P 44bff267e38e484880e2aefb7bc05807 [sys.catalog]: This master's current role is: LEADER
I20260812 08:03:36.986953 21502 sys_catalog.cc:458] T 00000000000000000000000000000000 P 44bff267e38e484880e2aefb7bc05807 [sys.catalog]: This master's current role is: LEADER
I20260812 08:03:36.987516 21506 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260812 08:03:36.988332 21506 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260812 08:03:36.988564 19841 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260812 08:03:36.990435 21506 catalog_manager.cc:1383] Generated new cluster ID: 0cf2143119154c5ca634c7e1b5860a25
I20260812 08:03:36.990496 21506 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260812 08:03:37.002079 21506 catalog_manager.cc:1406] Generated new certificate authority record
I20260812 08:03:37.002737 21506 catalog_manager.cc:1540] Loading token signing keys...
I20260812 08:03:37.013367 21506 catalog_manager.cc:6092] T 00000000000000000000000000000000 P 44bff267e38e484880e2aefb7bc05807: Generated new TSK 0
I20260812 08:03:37.013622 21506 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260812 08:03:37.021129 19841 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 08:03:37.023343 21519 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:03:37.023530 21523 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:03:37.023509 21520 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:03:37.023496 19841 server_base.cc:1061] running on GCE node
I20260812 08:03:37.023890 19841 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260812 08:03:37.023976 19841 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260812 08:03:37.024004 19841 hybrid_clock.cc:648] HybridClock initialized: now 1786521817024003 us; error 0 us; skew 500 ppm
I20260812 08:03:37.025015 19841 webserver.cc:533] Webserver started at http://127.19.96.65:36945/ using document root <none> and password file <none>
I20260812 08:03:37.025202 19841 fs_manager.cc:362] Metadata directory not provided
I20260812 08:03:37.025276 19841 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 08:03:37.025359 19841 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 08:03:37.025777 19841 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task3yToRy/test-tmp/prometheus_sd-itest.0.LocationLabelSDITest.TserverLocationLabel.1786521770778185-19841-0/minicluster-data/ts-0-root/instance:
uuid: "310694fad877413ca2d367ce7edd2a8d"
format_stamp: "Formatted at 2026-08-12 08:03:37 on dist-test-slave-8w9v"
I20260812 08:03:37.027295 19841 fs_manager.cc:696] Time spent creating directory manager: real 0.001s	user 0.002s	sys 0.000s
I20260812 08:03:37.028280 21529 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:03:37.028539 19841 fs_manager.cc:730] Time spent opening block manager: real 0.001s	user 0.001s	sys 0.000s
I20260812 08:03:37.028617 19841 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task3yToRy/test-tmp/prometheus_sd-itest.0.LocationLabelSDITest.TserverLocationLabel.1786521770778185-19841-0/minicluster-data/ts-0-root
uuid: "310694fad877413ca2d367ce7edd2a8d"
format_stamp: "Formatted at 2026-08-12 08:03:37 on dist-test-slave-8w9v"
I20260812 08:03:37.028676 19841 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task3yToRy/test-tmp/prometheus_sd-itest.0.LocationLabelSDITest.TserverLocationLabel.1786521770778185-19841-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-task3yToRy/test-tmp/prometheus_sd-itest.0.LocationLabelSDITest.TserverLocationLabel.1786521770778185-19841-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-task3yToRy/test-tmp/prometheus_sd-itest.0.LocationLabelSDITest.TserverLocationLabel.1786521770778185-19841-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 08:03:37.054217 19841 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:03:37.054639 19841 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:03:37.054929 19841 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260812 08:03:37.055460 19841 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 08:03:37.055501 19841 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:03:37.055570 19841 ts_tablet_manager.cc:616] Registered 0 tablets
I20260812 08:03:37.055614 19841 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:03:37.060518 19841 rpc_server.cc:307] RPC server started. Bound to: 127.19.96.65:33267
I20260812 08:03:37.060563 21612 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.19.96.65:33267 every 8 connection(s)
I20260812 08:03:37.061419 19841 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260812 08:03:37.063673 21620 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260812 08:03:37.063719 21621 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:03:37.063839 19841 server_base.cc:1061] running on GCE node
W20260812 08:03:37.063666 21623 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260812 08:03:37.064275 19841 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260812 08:03:37.064343 19841 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260812 08:03:37.064370 19841 hybrid_clock.cc:648] HybridClock initialized: now 1786521817064369 us; error 0 us; skew 500 ppm
I20260812 08:03:37.065415 19841 webserver.cc:533] Webserver started at http://127.19.96.66:36029/ using document root <none> and password file <none>
I20260812 08:03:37.065620 19841 fs_manager.cc:362] Metadata directory not provided
I20260812 08:03:37.065698 19841 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260812 08:03:37.065784 19841 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260812 08:03:37.066367 19841 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task3yToRy/test-tmp/prometheus_sd-itest.0.LocationLabelSDITest.TserverLocationLabel.1786521770778185-19841-0/minicluster-data/ts-1-root/instance:
uuid: "75b69f93012c44748b93c0473e9d8e0f"
format_stamp: "Formatted at 2026-08-12 08:03:37 on dist-test-slave-8w9v"
I20260812 08:03:37.066613 21613 heartbeater.cc:344] Connected to a master server at 127.19.96.126:44203
I20260812 08:03:37.066727 21613 heartbeater.cc:461] Registering TS with master...
I20260812 08:03:37.066946 21613 heartbeater.cc:507] Master 127.19.96.126:44203 requested a full tablet report, sending...
I20260812 08:03:37.074661 19841 fs_manager.cc:696] Time spent creating directory manager: real 0.008s	user 0.002s	sys 0.007s
I20260812 08:03:37.076525 21630 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:03:37.076874 19841 fs_manager.cc:730] Time spent opening block manager: real 0.001s	user 0.000s	sys 0.001s
I20260812 08:03:37.076973 19841 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task3yToRy/test-tmp/prometheus_sd-itest.0.LocationLabelSDITest.TserverLocationLabel.1786521770778185-19841-0/minicluster-data/ts-1-root
uuid: "75b69f93012c44748b93c0473e9d8e0f"
format_stamp: "Formatted at 2026-08-12 08:03:37 on dist-test-slave-8w9v"
I20260812 08:03:37.077072 19841 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task3yToRy/test-tmp/prometheus_sd-itest.0.LocationLabelSDITest.TserverLocationLabel.1786521770778185-19841-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-task3yToRy/test-tmp/prometheus_sd-itest.0.LocationLabelSDITest.TserverLocationLabel.1786521770778185-19841-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-task3yToRy/test-tmp/prometheus_sd-itest.0.LocationLabelSDITest.TserverLocationLabel.1786521770778185-19841-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260812 08:03:37.085652 19841 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260812 08:03:37.086233 19841 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260812 08:03:37.086649 19841 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260812 08:03:37.087323 19841 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260812 08:03:37.087386 19841 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:03:37.087445 19841 ts_tablet_manager.cc:616] Registered 0 tablets
I20260812 08:03:37.087540 19841 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260812 08:03:37.094508 19841 rpc_server.cc:307] RPC server started. Bound to: 127.19.96.66:39119
I20260812 08:03:37.094532 21699 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.19.96.66:39119 every 8 connection(s)
I20260812 08:03:37.100629 21700 heartbeater.cc:344] Connected to a master server at 127.19.96.126:44203
I20260812 08:03:37.100785 21700 heartbeater.cc:461] Registering TS with master...
I20260812 08:03:37.101051 21700 heartbeater.cc:507] Master 127.19.96.126:44203 requested a full tablet report, sending...
I20260812 08:03:37.126284 21456 ts_manager.cc:194] Registered new tserver with Master: 310694fad877413ca2d367ce7edd2a8d (127.19.96.65:33267)
I20260812 08:03:37.127754 21456 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:37316
I20260812 08:03:37.145949 21455 ts_manager.cc:194] Registered new tserver with Master: 75b69f93012c44748b93c0473e9d8e0f (127.19.96.66:39119)
I20260812 08:03:37.146502 19841 internal_mini_cluster.cc:371] 2 TS(s) registered with all masters after 0.05125213s
I20260812 08:03:37.146615 21455 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:37330
I20260812 08:03:37.147439 19841 internal_mini_cluster.cc:371] 2 TS(s) registered with all masters after 5.4679e-05s
time=2026-08-12T08:03:37.217Z level=INFO source=main.go:1678 msg="updated GOGC" old=100 new=75
time=2026-08-12T08:03:37.218Z level=INFO source=main.go:744 msg="Leaving GOMAXPROCS=4: CPU quota undefined" component=automaxprocs
time=2026-08-12T08:03:37.219Z level=INFO source=memlimit.go:198 msg="GOMEMLIMIT is updated" component=automemlimit package=github.com/KimMachineGun/automemlimit/memlimit GOMEMLIMIT=14142552883 previous=9223372036854775807
time=2026-08-12T08:03:37.219Z level=INFO source=main.go:851 msg="Starting Prometheus Server" mode=server version="(version=3.11.2, branch=HEAD, revision=f0f0fdd679dcd6df320b0558b20919f7cd44c407)"
time=2026-08-12T08:03:37.219Z level=INFO source=main.go:856 msg="operational information" build_context="(go=go1.26.2, platform=linux/amd64, user=root@d2684485c347, date=20260413-11:53:40, tags=netgo,builtinassets)" host_details="(Linux 5.4.0-1157-gcp #166~18.04.1-Ubuntu SMP Wed Jan 21 01:57:46 UTC 2026 x86_64 dist-test-slave-8w9v (none))" fd_limits="(soft=1048576, hard=1048576)" vm_limits="(soft=unlimited, hard=unlimited)"
time=2026-08-12T08:03:37.437Z level=INFO source=web.go:710 msg="Start listening for connections" component=web address=127.0.0.1:0
time=2026-08-12T08:03:37.438Z level=INFO source=main.go:1410 msg="Starting TSDB ..."
time=2026-08-12T08:03:37.447Z level=INFO source=head.go:698 msg="Replaying on-disk memory mappable chunks if any" component=tsdb
time=2026-08-12T08:03:37.447Z level=INFO source=head.go:784 msg="On-disk memory mappable chunks replay completed" component=tsdb duration=2.35µs
time=2026-08-12T08:03:37.447Z level=INFO source=head.go:792 msg="Replaying WAL, this may take a while" component=tsdb
time=2026-08-12T08:03:37.449Z level=INFO source=tls_config.go:354 msg="Listening on" component=web address=127.0.0.1:36679
time=2026-08-12T08:03:37.449Z level=INFO source=tls_config.go:357 msg="TLS is disabled." component=web http2=false address=127.0.0.1:36679
time=2026-08-12T08:03:37.450Z level=INFO source=head.go:865 msg="WAL segment loaded" component=tsdb segment=0 maxSegment=0 duration=2.574026ms
time=2026-08-12T08:03:37.450Z level=INFO source=head.go:902 msg="WAL replay completed" component=tsdb checkpoint_replay_duration=73.444µs wal_replay_duration=2.637262ms wbl_replay_duration=175ns chunk_snapshot_load_duration=0s mmap_chunk_replay_duration=2.35µs total_replay_duration=2.756393ms
time=2026-08-12T08:03:37.454Z level=INFO source=main.go:1431 msg="filesystem information" fs_type=OVERLAYFS_SUPER_MAGIC
time=2026-08-12T08:03:37.454Z level=INFO source=main.go:1434 msg="TSDB started"
time=2026-08-12T08:03:37.454Z level=INFO source=main.go:1632 msg="Loading configuration file" filename=/tmp/dist-test-task3yToRy/test-tmp/prometheus_sd-itest.0.LocationLabelSDITest.TserverLocationLabel.1786521770778185-19841-0/prometheus.yml
time=2026-08-12T08:03:37.454Z level=INFO source=main.go:1048 msg="TSDB retention updated" duration=15d size=0B percentage=0
time=2026-08-12T08:03:37.458Z level=INFO source=main.go:1671 msg="Completed loading of configuration file" db_storage=41.808µs remote_storage=2.623µs web_handler=868ns query_engine=1.056µs scrape=3.865895ms scrape_sd=341.204µs notify=15.568µs notify_sd=2.433µs rules=3.203µs tracing=172.644µs filename=/tmp/dist-test-task3yToRy/test-tmp/prometheus_sd-itest.0.LocationLabelSDITest.TserverLocationLabel.1786521770778185-19841-0/prometheus.yml totalDuration=4.777954ms
time=2026-08-12T08:03:37.459Z level=INFO source=main.go:1395 msg="Server is ready to receive web requests."
time=2026-08-12T08:03:37.459Z level=INFO source=manager.go:209 msg="Starting rule manager..." component="rule manager"
I20260812 08:03:38.129345 21613 heartbeater.cc:499] Master 127.19.96.126:44203 was elected leader, sending a full tablet report...
I20260812 08:03:38.148046 21700 heartbeater.cc:499] Master 127.19.96.126:44203 was elected leader, sending a full tablet report...
time=2026-08-12T08:03:44.260Z level=WARN source=main.go:1191 msg="Received an OS signal, exiting gracefully..." signal=terminated
time=2026-08-12T08:03:44.261Z level=INFO source=main.go:1216 msg="Stopping scrape discovery manager..."
time=2026-08-12T08:03:44.261Z level=INFO source=main.go:1230 msg="Stopping notify discovery manager..."
time=2026-08-12T08:03:44.261Z level=INFO source=main.go:1245 msg="Stopping rule manager manager..."
time=2026-08-12T08:03:44.261Z level=INFO source=manager.go:223 msg="Stopping rule manager..." component="rule manager"
time=2026-08-12T08:03:44.261Z level=INFO source=manager.go:239 msg="Rule manager stopped" component="rule manager"
time=2026-08-12T08:03:44.261Z level=INFO source=main.go:1269 msg="Stopping scrape manager..."
time=2026-08-12T08:03:44.264Z level=INFO source=main.go:1241 msg="Rule manager stopped"
time=2026-08-12T08:03:44.264Z level=INFO source=main.go:1226 msg="Notify discovery manager stopped"
time=2026-08-12T08:03:44.264Z level=INFO source=main.go:1284 msg="Stopping tracing manager..."
time=2026-08-12T08:03:44.264Z level=INFO source=main.go:1369 msg="Stopping reloaders..."
time=2026-08-12T08:03:44.264Z level=INFO source=main.go:1458 msg="Stopping storage..."
time=2026-08-12T08:03:44.264Z level=INFO source=main.go:1261 msg="Scrape manager stopped"
time=2026-08-12T08:03:44.264Z level=INFO source=main.go:1361 msg="Reloaders stopped"
time=2026-08-12T08:03:44.264Z level=INFO source=main.go:1280 msg="Tracing manager stopped"
time=2026-08-12T08:03:44.264Z level=INFO source=main.go:1212 msg="Scrape discovery manager stopped"
time=2026-08-12T08:03:44.269Z level=INFO source=main.go:1536 msg="Stopping web handler..."
time=2026-08-12T08:03:44.269Z level=INFO source=main.go:1559 msg="Stopping notifier manager..."
time=2026-08-12T08:03:44.269Z level=INFO source=manager.go:321 msg="Stopping notification manager..." component=notifier
time=2026-08-12T08:03:44.269Z level=INFO source=main.go:1454 msg="TSDB stopped"
time=2026-08-12T08:03:44.269Z level=INFO source=main.go:1555 msg="Notifier manager stopped"
time=2026-08-12T08:03:44.269Z level=INFO source=main.go:1532 msg="Web handler stopped"
time=2026-08-12T08:03:44.269Z level=INFO source=main.go:1570 msg="See you next time!"
I20260812 08:03:44.281454 19841 tablet_server.cc:179] TabletServer@127.19.96.65:0 shutting down...
I20260812 08:03:44.285926 19841 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260812 08:03:44.298380 19841 tablet_server.cc:196] TabletServer@127.19.96.65:0 shutdown complete.
I20260812 08:03:44.300684 19841 tablet_server.cc:179] TabletServer@127.19.96.66:0 shutting down...
I20260812 08:03:44.305600 19841 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260812 08:03:44.317975 19841 tablet_server.cc:196] TabletServer@127.19.96.66:0 shutdown complete.
I20260812 08:03:44.319960 19841 master.cc:562] Master@127.19.96.126:44203 shutting down...
I20260812 08:03:44.323499 19841 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 44bff267e38e484880e2aefb7bc05807 [term 1 LEADER]: Raft consensus shutting down.
I20260812 08:03:44.323699 19841 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 44bff267e38e484880e2aefb7bc05807 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260812 08:03:44.323791 19841 tablet_replica.cc:333] T 00000000000000000000000000000000 P 44bff267e38e484880e2aefb7bc05807: stopping tablet replica
I20260812 08:03:44.336437 19841 master.cc:584] Master@127.19.96.126:44203 shutdown complete.
[       OK ] LocationLabelSDITest.TserverLocationLabel (7392 ms)
[----------] 1 test from LocationLabelSDITest (7392 ms total)

[----------] Global test environment tear-down
[==========] 6 tests from 3 test suites ran. (53555 ms total)
[  PASSED  ] 6 tests.
I20260812 08:03:44.342599 19841 logging.cc:436] LogThrottler /workspace/apache/dev/local/kudu/src/kudu/tablet/tablet.cc:2406: suppressed but not reported on 1 messages since previous log ~24 seconds ago
I20260812 08:03:44.342676 19841 logging.cc:436] LogThrottler /workspace/apache/dev/local/kudu/src/kudu/rpc/proxy.cc:239: suppressed but not reported on 1 messages since previous log ~24 seconds ago
