[2026-04-15 22:42:13,250][  INFO][PID:2787499] Marking build as starting
[2026-04-15 22:42:13,458][  INFO][PID:2787499] Checking for cancel request
[2026-04-15 22:42:13,464][  INFO][PID:2787499] VM allocation process starts
[2026-04-15 22:42:13,553][  INFO][PID:2787499] Trying to allocate VM: ResallocHost, ticket_id=3190, requested_tags=['arch_noarch', 'copr_builder']
[2026-04-15 22:42:18,571][  INFO][PID:2787499] Checking for cancel request
[2026-04-15 22:42:23,573][  INFO][PID:2787499] Checking for cancel request
[2026-04-15 22:42:28,575][  INFO][PID:2787499] Checking for cancel request
[2026-04-15 22:42:33,577][  INFO][PID:2787499] Checking for cancel request
[2026-04-15 22:42:38,580][  INFO][PID:2787499] Checking for cancel request
[2026-04-15 22:42:43,584][  INFO][PID:2787499] Checking for cancel request
[2026-04-15 22:42:48,586][  INFO][PID:2787499] Checking for cancel request
[2026-04-15 22:42:53,593][  INFO][PID:2787499] Checking for cancel request
[2026-04-15 22:42:58,598][  INFO][PID:2787499] Checking for cancel request
[2026-04-15 22:43:03,601][  INFO][PID:2787499] Checking for cancel request
[2026-04-15 22:43:08,603][  INFO][PID:2787499] Checking for cancel request
[2026-04-15 22:43:13,605][  INFO][PID:2787499] Checking for cancel request
[2026-04-15 22:43:18,610][  INFO][PID:2787499] Checking for cancel request
[2026-04-15 22:43:23,612][  INFO][PID:2787499] Checking for cancel request
[2026-04-15 22:43:28,615][  INFO][PID:2787499] Checking for cancel request
[2026-04-15 22:43:33,619][  INFO][PID:2787499] Checking for cancel request
[2026-04-15 22:43:38,624][  INFO][PID:2787499] Checking for cancel request
[2026-04-15 22:43:43,626][  INFO][PID:2787499] Checking for cancel request
[2026-04-15 22:43:48,634][  INFO][PID:2787499] Checking for cancel request
[2026-04-15 22:43:53,638][  INFO][PID:2787499] Checking for cancel request
[2026-04-15 22:43:58,639][  INFO][PID:2787499] Checking for cancel request
[2026-04-15 22:44:03,643][  INFO][PID:2787499] Checking for cancel request
[2026-04-15 22:44:08,654][  INFO][PID:2787499] Checking for cancel request
[2026-04-15 22:44:13,659][  INFO][PID:2787499] Checking for cancel request
[2026-04-15 22:44:18,663][  INFO][PID:2787499] Checking for cancel request
[2026-04-15 22:44:23,666][  INFO][PID:2787499] Checking for cancel request
[2026-04-15 22:44:28,669][  INFO][PID:2787499] Checking for cancel request
[2026-04-15 22:44:33,670][  INFO][PID:2787499] Checking for cancel request
[2026-04-15 22:44:38,676][  INFO][PID:2787499] Checking for cancel request
[2026-04-15 22:44:43,677][  INFO][PID:2787499] Checking for cancel request
[2026-04-15 22:44:48,679][  INFO][PID:2787499] Checking for cancel request
[2026-04-15 22:44:53,681][  INFO][PID:2787499] Checking for cancel request
[2026-04-15 22:44:58,687][  INFO][PID:2787499] Checking for cancel request
[2026-04-15 22:45:03,691][  INFO][PID:2787499] Checking for cancel request
[2026-04-15 22:45:08,700][  INFO][PID:2787499] Checking for cancel request
[2026-04-15 22:45:13,703][  INFO][PID:2787499] Checking for cancel request
[2026-04-15 22:45:18,707][  INFO][PID:2787499] Checking for cancel request
[2026-04-15 22:45:23,710][  INFO][PID:2787499] Checking for cancel request
[2026-04-15 22:45:28,713][  INFO][PID:2787499] Checking for cancel request
[2026-04-15 22:45:31,951][  INFO][PID:2787499] Allocated host ResallocHost, ticket_id=3190, hostname=100.31.237.99, name=aws_x86_64_spot_dev_02147494_20260415_224326, requested_tags=['arch_noarch', 'copr_builder']
[2026-04-15 22:45:31,953][  INFO][PID:2787499] Allocating ssh connection to builder
[2026-04-15 22:45:31,954][  INFO][PID:2787499] Checking that builder machine is OK
[2026-04-15 22:45:32,383][  INFO][PID:2787499] Installed copr-rpmbuild version: 1.6
[2026-04-15 22:45:32,385][ ERROR][PID:2787499] Re-try request for task on 'ResallocHost, ticket_id=3190, hostname=100.31.237.99, name=aws_x86_64_spot_dev_02147494_20260415_224326, requested_tags=['arch_noarch', 'copr_builder']': Minimum version for builder is 1.6.1
[2026-04-15 22:45:32,387][  INFO][PID:2787499] VM Release request
[2026-04-15 22:45:32,451][  INFO][PID:2787499] Retry #1 (on other host)
[2026-04-15 22:45:32,453][  INFO][PID:2787499] Checking for cancel request
[2026-04-15 22:45:32,456][  INFO][PID:2787499] VM allocation process starts
[2026-04-15 22:45:32,522][  INFO][PID:2787499] Trying to allocate VM: ResallocHost, ticket_id=3250, requested_tags=['arch_noarch', 'copr_builder']
[2026-04-15 22:45:37,526][  INFO][PID:2787499] Checking for cancel request
[2026-04-15 22:45:42,540][  INFO][PID:2787499] Checking for cancel request
[2026-04-15 22:45:47,551][  INFO][PID:2787499] Checking for cancel request
[2026-04-15 22:45:52,564][  INFO][PID:2787499] Checking for cancel request
[2026-04-15 22:45:57,569][  INFO][PID:2787499] Checking for cancel request
[2026-04-15 22:46:02,573][  INFO][PID:2787499] Checking for cancel request
[2026-04-15 22:46:07,579][  INFO][PID:2787499] Checking for cancel request
[2026-04-15 22:46:12,581][  INFO][PID:2787499] Checking for cancel request
[2026-04-15 22:46:17,589][  INFO][PID:2787499] Checking for cancel request
[2026-04-15 22:46:22,871][  INFO][PID:2787499] Checking for cancel request
[2026-04-15 22:46:27,882][  INFO][PID:2787499] Checking for cancel request
[2026-04-15 22:46:32,890][  INFO][PID:2787499] Checking for cancel request
[2026-04-15 22:46:37,902][  INFO][PID:2787499] Checking for cancel request
[2026-04-15 22:46:42,915][  INFO][PID:2787499] Checking for cancel request
[2026-04-15 22:46:47,925][  INFO][PID:2787499] Checking for cancel request
[2026-04-15 22:46:52,939][  INFO][PID:2787499] Checking for cancel request
[2026-04-15 22:46:57,943][  INFO][PID:2787499] Checking for cancel request
[2026-04-15 22:47:02,945][  INFO][PID:2787499] Checking for cancel request
[2026-04-15 22:47:07,957][  INFO][PID:2787499] Checking for cancel request
[2026-04-15 22:47:12,965][  INFO][PID:2787499] Checking for cancel request
[2026-04-15 22:47:17,969][  INFO][PID:2787499] Checking for cancel request
[2026-04-15 22:47:22,981][  INFO][PID:2787499] Checking for cancel request
[2026-04-15 22:47:27,982][  INFO][PID:2787499] Checking for cancel request
[2026-04-15 22:47:32,985][  INFO][PID:2787499] Checking for cancel request
[2026-04-15 22:47:37,992][  INFO][PID:2787499] Checking for cancel request
[2026-04-15 22:47:42,997][  INFO][PID:2787499] Checking for cancel request
[2026-04-15 22:47:48,009][  INFO][PID:2787499] Checking for cancel request
[2026-04-15 22:47:50,891][  INFO][PID:2787499] Allocated host ResallocHost, ticket_id=3250, hostname=100.53.241.233, name=aws_x86_64_spot_dev_02147504_20260415_224535, requested_tags=['arch_noarch', 'copr_builder']
[2026-04-15 22:47:50,892][  INFO][PID:2787499] Allocating ssh connection to builder
[2026-04-15 22:47:50,893][  INFO][PID:2787499] Checking that builder machine is OK
[2026-04-15 22:47:51,257][  INFO][PID:2787499] Installed copr-rpmbuild version: 1.6
[2026-04-15 22:47:51,259][ ERROR][PID:2787499] Re-try request for task on 'ResallocHost, ticket_id=3250, hostname=100.53.241.233, name=aws_x86_64_spot_dev_02147504_20260415_224535, requested_tags=['arch_noarch', 'copr_builder']': Minimum version for builder is 1.6.1
[2026-04-15 22:47:51,259][  INFO][PID:2787499] VM Release request
[2026-04-15 22:47:51,286][  INFO][PID:2787499] Retry #2 (on other host)
[2026-04-15 22:47:51,287][  INFO][PID:2787499] Checking for cancel request
[2026-04-15 22:47:51,289][  INFO][PID:2787499] VM allocation process starts
[2026-04-15 22:47:51,322][  INFO][PID:2787499] Trying to allocate VM: ResallocHost, ticket_id=3267, requested_tags=['arch_noarch', 'copr_builder']
[2026-04-15 22:47:54,381][  INFO][PID:2787499] Allocated host ResallocHost, ticket_id=3267, hostname=100.53.241.233, name=aws_x86_64_spot_dev_02147504_20260415_224535, requested_tags=['arch_noarch', 'copr_builder']
[2026-04-15 22:47:54,383][  INFO][PID:2787499] Allocating ssh connection to builder
[2026-04-15 22:47:54,386][  INFO][PID:2787499] Checking that builder machine is OK
[2026-04-15 22:47:54,449][  INFO][PID:2787499] Installed copr-rpmbuild version: 1.6
[2026-04-15 22:47:54,451][ ERROR][PID:2787499] Re-try request for task on 'ResallocHost, ticket_id=3267, hostname=100.53.241.233, name=aws_x86_64_spot_dev_02147504_20260415_224535, requested_tags=['arch_noarch', 'copr_builder']': Minimum version for builder is 1.6.1
[2026-04-15 22:47:54,452][  INFO][PID:2787499] VM Release request
[2026-04-15 22:47:54,484][  INFO][PID:2787499] Retry #3 (on other host)
[2026-04-15 22:47:54,486][  INFO][PID:2787499] Checking for cancel request
[2026-04-15 22:47:54,487][  INFO][PID:2787499] VM allocation process starts
[2026-04-15 22:47:54,516][  INFO][PID:2787499] Trying to allocate VM: ResallocHost, ticket_id=3269, requested_tags=['arch_noarch', 'copr_builder']
[2026-04-15 22:47:59,527][  INFO][PID:2787499] Checking for cancel request
[2026-04-15 22:48:04,539][  INFO][PID:2787499] Checking for cancel request
[2026-04-15 22:48:09,549][  INFO][PID:2787499] Checking for cancel request
[2026-04-15 22:48:14,557][  INFO][PID:2787499] Checking for cancel request
[2026-04-15 22:48:19,595][  INFO][PID:2787499] Checking for cancel request
[2026-04-15 22:48:24,606][  INFO][PID:2787499] Checking for cancel request
[2026-04-15 22:48:29,623][  INFO][PID:2787499] Checking for cancel request
[2026-04-15 22:48:34,626][  INFO][PID:2787499] Checking for cancel request
[2026-04-15 22:48:39,633][  INFO][PID:2787499] Checking for cancel request
[2026-04-15 22:48:44,639][  INFO][PID:2787499] Checking for cancel request
[2026-04-15 22:48:49,643][  INFO][PID:2787499] Checking for cancel request
[2026-04-15 22:48:54,655][  INFO][PID:2787499] Checking for cancel request
[2026-04-15 22:48:59,665][  INFO][PID:2787499] Checking for cancel request
[2026-04-15 22:49:04,680][  INFO][PID:2787499] Checking for cancel request
[2026-04-15 22:49:09,682][  INFO][PID:2787499] Checking for cancel request
[2026-04-15 22:49:14,693][  INFO][PID:2787499] Checking for cancel request
[2026-04-15 22:49:19,703][  INFO][PID:2787499] Checking for cancel request
[2026-04-15 22:49:23,006][  INFO][PID:2787499] Allocated host ResallocHost, ticket_id=3269, hostname=2620:52:6:1161:dead:beef:cafe:c905, name=vmhost_x86_01_dev_02147511_20260415_224609, requested_tags=['arch_noarch', 'copr_builder']
[2026-04-15 22:49:23,007][  INFO][PID:2787499] Allocating ssh connection to builder
[2026-04-15 22:49:23,013][  INFO][PID:2787499] Checking that builder machine is OK
[2026-04-15 22:49:23,737][  INFO][PID:2787499] Installed copr-rpmbuild version: 1.6
[2026-04-15 22:49:23,743][ ERROR][PID:2787499] Re-try request for task on 'ResallocHost, ticket_id=3269, hostname=2620:52:6:1161:dead:beef:cafe:c905, name=vmhost_x86_01_dev_02147511_20260415_224609, requested_tags=['arch_noarch', 'copr_builder']': Minimum version for builder is 1.6.1
[2026-04-15 22:49:23,746][  INFO][PID:2787499] VM Release request
[2026-04-15 22:49:23,807][ ERROR][PID:2787499] Backend process error: Three host tried without success: {'100.31.237.99', '2620:52:6:1161:dead:beef:cafe:c905', '100.53.241.233'}
[2026-04-15 22:49:23,815][WARNING][PID:2787499] Switching not-finished job state to 'failed'
[2026-04-15 22:49:23,817][  INFO][PID:2787499] Worker failed build, took 430.58861565589905
[2026-04-15 22:49:23,825][  INFO][PID:2787499] Sending build state back to frontend: {
    "builds": [
        {
            "timeout": 108000,
            "frontend_base_url": "https://copr.stg.fedoraproject.org",
            "memory_reqs": null,
            "enable_net": true,
            "project_owner": "@copr",
            "project_name": "TEST1776291979768663993-build-batches",
            "project_dirname": "TEST1776291979768663993-build-batches",
            "submitter": "frostyx",
            "ended_on": 1776293363.8150744,
            "started_on": 1776292933.2264588,
            "submitted_on": null,
            "status": 0,
            "chroot": "srpm-builds",
            "arch": null,
            "buildroot_pkgs": null,
            "task_id": "2925227",
            "build_id": 2925227,
            "package_name": null,
            "package_version": null,
            "git_repo": null,
            "git_hash": null,
            "git_branch": null,
            "source_type": 1,
            "source_json": "{\"url\": \"https://github.com/fedora-copr/copr-test-sources/raw/main/hello-2.8-1.src.rpm\"}",
            "pkg_name": null,
            "pkg_main_version": null,
            "pkg_epoch": null,
            "pkg_release": null,
            "srpm_url": null,
            "uses_devel_repo": null,
            "sandbox": "@copr/TEST1776291979768663993-build-batches--frostyx",
            "results": null,
            "appstream": false,
            "allow_user_ssh": null,
            "ssh_public_keys": null,
            "storage": null,
            "repos": [],
            "background": false,
            "distributions_in_build": [],
            "distributions_in_project": [
                "fedora-43"
            ],
            "destdir": "/var/lib/copr/public_html/results/@copr/TEST1776291979768663993-build-batches",
            "results_repo_url": "https://download.copr-dev.fedorainfracloud.org/results/@copr/TEST1776291979768663993-build-batches",
            "result_dir": "02925227",
            "built_packages": "",
            "tags": [
                "arch_noarch"
            ],
            "id": 2925227,
            "mockchain_macros": {
                "copr_username": "@copr",
                "copr_projectname": "TEST1776291979768663993-build-batches",
                "vendor": "Fedora Project COPR (@copr/TEST1776291979768663993-build-batches)"
            }
        }
    ]
}
[2026-04-15 22:49:24,075][  INFO][PID:2787499] Sending fedora-messaging bus message in build.end
[2026-04-15 22:49:29,557][  INFO][PID:2787499] Compressing /var/lib/copr/public_html/results/@copr/TEST1776291979768663993-build-batches/srpm-builds/02925227/builder-live.log by gzip
[2026-04-15 22:49:29,562][  INFO][PID:2787499] Running command 'gzip /var/lib/copr/public_html/results/@copr/TEST1776291979768663993-build-batches/srpm-builds/02925227/builder-live.log' as PID 2804107
[2026-04-15 22:49:29,571][  INFO][PID:2787499] Finished after 0 seconds with exit code 1 (gzip /var/lib/copr/public_html/results/@copr/TEST1776291979768663993-build-batches/srpm-builds/02925227/builder-live.log)
stdout:

stderr:
gzip: /var/lib/copr/public_html/results/@copr/TEST1776291979768663993-build-batches/srpm-builds/02925227/builder-live.log: No such file or directory


[2026-04-15 22:49:29,577][ ERROR][PID:2787499] Unable to compress file /var/lib/copr/public_html/results/@copr/TEST1776291979768663993-build-batches/srpm-builds/02925227/builder-live.log
[2026-04-15 22:49:29,584][  INFO][PID:2787499] Compressing /var/lib/copr/public_html/results/@copr/TEST1776291979768663993-build-batches/srpm-builds/02925227/backend.log by gzip
[2026-04-15 22:49:29,600][  INFO][PID:2787499] Running command 'gzip /var/lib/copr/public_html/results/@copr/TEST1776291979768663993-build-batches/srpm-builds/02925227/backend.log' as PID 2804108