[2026-04-15 22:41:26,577][  INFO][PID:2785715] Marking build as starting
[2026-04-15 22:41:26,650][  INFO][PID:2785715] Checking for cancel request
[2026-04-15 22:41:26,654][  INFO][PID:2785715] VM allocation process starts
[2026-04-15 22:41:26,683][  INFO][PID:2785715] Trying to allocate VM: ResallocHost, ticket_id=3171, requested_tags=['arch_noarch', 'copr_builder']
[2026-04-15 22:41:31,687][  INFO][PID:2785715] Checking for cancel request
[2026-04-15 22:41:36,693][  INFO][PID:2785715] Checking for cancel request
[2026-04-15 22:41:41,697][  INFO][PID:2785715] Checking for cancel request
[2026-04-15 22:41:46,700][  INFO][PID:2785715] Checking for cancel request
[2026-04-15 22:41:51,704][  INFO][PID:2785715] Checking for cancel request
[2026-04-15 22:41:56,709][  INFO][PID:2785715] Checking for cancel request
[2026-04-15 22:42:01,711][  INFO][PID:2785715] Checking for cancel request
[2026-04-15 22:42:06,715][  INFO][PID:2785715] Checking for cancel request
[2026-04-15 22:42:11,727][  INFO][PID:2785715] Checking for cancel request
[2026-04-15 22:42:16,731][  INFO][PID:2785715] Checking for cancel request
[2026-04-15 22:42:21,732][  INFO][PID:2785715] Checking for cancel request
[2026-04-15 22:42:26,738][  INFO][PID:2785715] Checking for cancel request
[2026-04-15 22:42:31,740][  INFO][PID:2785715] Checking for cancel request
[2026-04-15 22:42:36,741][  INFO][PID:2785715] Checking for cancel request
[2026-04-15 22:42:41,744][  INFO][PID:2785715] Checking for cancel request
[2026-04-15 22:42:46,745][  INFO][PID:2785715] Checking for cancel request
[2026-04-15 22:42:51,747][  INFO][PID:2785715] Checking for cancel request
[2026-04-15 22:42:56,750][  INFO][PID:2785715] Checking for cancel request
[2026-04-15 22:43:01,753][  INFO][PID:2785715] Checking for cancel request
[2026-04-15 22:43:06,757][  INFO][PID:2785715] Checking for cancel request
[2026-04-15 22:43:11,761][  INFO][PID:2785715] Checking for cancel request
[2026-04-15 22:43:16,765][  INFO][PID:2785715] Checking for cancel request
[2026-04-15 22:43:21,768][  INFO][PID:2785715] Checking for cancel request
[2026-04-15 22:43:26,770][  INFO][PID:2785715] Checking for cancel request
[2026-04-15 22:43:31,777][  INFO][PID:2785715] Checking for cancel request
[2026-04-15 22:43:36,792][  INFO][PID:2785715] Checking for cancel request
[2026-04-15 22:43:41,809][  INFO][PID:2785715] Checking for cancel request
[2026-04-15 22:43:46,823][  INFO][PID:2785715] Checking for cancel request
[2026-04-15 22:43:51,832][  INFO][PID:2785715] Checking for cancel request
[2026-04-15 22:43:56,835][  INFO][PID:2785715] Checking for cancel request
[2026-04-15 22:44:01,837][  INFO][PID:2785715] Checking for cancel request
[2026-04-15 22:44:06,842][  INFO][PID:2785715] Checking for cancel request
[2026-04-15 22:44:11,845][  INFO][PID:2785715] Checking for cancel request
[2026-04-15 22:44:16,846][  INFO][PID:2785715] Checking for cancel request
[2026-04-15 22:44:21,850][  INFO][PID:2785715] Checking for cancel request
[2026-04-15 22:44:26,853][  INFO][PID:2785715] Checking for cancel request
[2026-04-15 22:44:31,856][  INFO][PID:2785715] Checking for cancel request
[2026-04-15 22:44:36,858][  INFO][PID:2785715] Checking for cancel request
[2026-04-15 22:44:41,860][  INFO][PID:2785715] Checking for cancel request
[2026-04-15 22:44:46,863][  INFO][PID:2785715] Checking for cancel request
[2026-04-15 22:44:51,865][  INFO][PID:2785715] Checking for cancel request
[2026-04-15 22:44:56,870][  INFO][PID:2785715] Checking for cancel request
[2026-04-15 22:45:01,878][  INFO][PID:2785715] Checking for cancel request
[2026-04-15 22:45:06,881][  INFO][PID:2785715] Checking for cancel request
[2026-04-15 22:45:11,888][  INFO][PID:2785715] Checking for cancel request
[2026-04-15 22:45:16,894][  INFO][PID:2785715] Checking for cancel request
[2026-04-15 22:45:21,900][  INFO][PID:2785715] Checking for cancel request
[2026-04-15 22:45:26,903][  INFO][PID:2785715] Checking for cancel request
[2026-04-15 22:45:31,908][  INFO][PID:2785715] Checking for cancel request
[2026-04-15 22:45:36,909][  INFO][PID:2785715] Checking for cancel request
[2026-04-15 22:45:41,913][  INFO][PID:2785715] Checking for cancel request
[2026-04-15 22:45:44,941][  INFO][PID:2785715] Allocated host ResallocHost, ticket_id=3171, hostname=2620:52:6:1161:dead:beef:cafe:c903, name=vmhost_p09_04_dev_02147487_20260415_224136, requested_tags=['arch_noarch', 'copr_builder']
[2026-04-15 22:45:44,942][  INFO][PID:2785715] Allocating ssh connection to builder
[2026-04-15 22:45:44,946][  INFO][PID:2785715] Checking that builder machine is OK
[2026-04-15 22:45:46,009][  INFO][PID:2785715] Installed copr-rpmbuild version: 1.6
[2026-04-15 22:45:46,017][ ERROR][PID:2785715] Re-try request for task on 'ResallocHost, ticket_id=3171, hostname=2620:52:6:1161:dead:beef:cafe:c903, name=vmhost_p09_04_dev_02147487_20260415_224136, requested_tags=['arch_noarch', 'copr_builder']': Minimum version for builder is 1.6.1
[2026-04-15 22:45:46,018][  INFO][PID:2785715] VM Release request
[2026-04-15 22:45:46,066][  INFO][PID:2785715] Retry #1 (on other host)
[2026-04-15 22:45:46,068][  INFO][PID:2785715] Checking for cancel request
[2026-04-15 22:45:46,078][  INFO][PID:2785715] VM allocation process starts
[2026-04-15 22:45:46,132][  INFO][PID:2785715] Trying to allocate VM: ResallocHost, ticket_id=3257, requested_tags=['arch_noarch', 'copr_builder']
[2026-04-15 22:45:51,146][  INFO][PID:2785715] Checking for cancel request
[2026-04-15 22:45:52,265][  INFO][PID:2785715] Allocated host ResallocHost, ticket_id=3257, hostname=2620:52:6:1161:dead:beef:cafe:c903, name=vmhost_p09_04_dev_02147487_20260415_224136, requested_tags=['arch_noarch', 'copr_builder']
[2026-04-15 22:45:52,267][  INFO][PID:2785715] Allocating ssh connection to builder
[2026-04-15 22:45:52,268][  INFO][PID:2785715] Checking that builder machine is OK
[2026-04-15 22:45:52,454][  INFO][PID:2785715] Installed copr-rpmbuild version: 1.6
[2026-04-15 22:45:52,457][ ERROR][PID:2785715] Re-try request for task on 'ResallocHost, ticket_id=3257, hostname=2620:52:6:1161:dead:beef:cafe:c903, name=vmhost_p09_04_dev_02147487_20260415_224136, requested_tags=['arch_noarch', 'copr_builder']': Minimum version for builder is 1.6.1
[2026-04-15 22:45:52,458][  INFO][PID:2785715] VM Release request
[2026-04-15 22:45:52,517][  INFO][PID:2785715] Retry #2 (on other host)
[2026-04-15 22:45:52,520][  INFO][PID:2785715] Checking for cancel request
[2026-04-15 22:45:52,529][  INFO][PID:2785715] VM allocation process starts
[2026-04-15 22:45:52,580][  INFO][PID:2785715] Trying to allocate VM: ResallocHost, ticket_id=3259, requested_tags=['arch_noarch', 'copr_builder']
[2026-04-15 22:45:57,593][  INFO][PID:2785715] Checking for cancel request
[2026-04-15 22:46:02,596][  INFO][PID:2785715] Checking for cancel request
[2026-04-15 22:46:07,602][  INFO][PID:2785715] Checking for cancel request
[2026-04-15 22:46:12,606][  INFO][PID:2785715] Checking for cancel request
[2026-04-15 22:46:17,611][  INFO][PID:2785715] Checking for cancel request
[2026-04-15 22:46:22,870][  INFO][PID:2785715] Checking for cancel request
[2026-04-15 22:46:27,883][  INFO][PID:2785715] Checking for cancel request
[2026-04-15 22:46:32,892][  INFO][PID:2785715] Checking for cancel request
[2026-04-15 22:46:37,902][  INFO][PID:2785715] Checking for cancel request
[2026-04-15 22:46:42,916][  INFO][PID:2785715] Checking for cancel request
[2026-04-15 22:46:47,928][  INFO][PID:2785715] Checking for cancel request
[2026-04-15 22:46:52,936][  INFO][PID:2785715] Checking for cancel request
[2026-04-15 22:46:57,947][  INFO][PID:2785715] Checking for cancel request
[2026-04-15 22:47:02,948][  INFO][PID:2785715] Checking for cancel request
[2026-04-15 22:47:07,964][  INFO][PID:2785715] Checking for cancel request
[2026-04-15 22:47:12,972][  INFO][PID:2785715] Checking for cancel request
[2026-04-15 22:47:17,980][  INFO][PID:2785715] Checking for cancel request
[2026-04-15 22:47:22,987][  INFO][PID:2785715] Checking for cancel request
[2026-04-15 22:47:27,994][  INFO][PID:2785715] Checking for cancel request
[2026-04-15 22:47:33,003][  INFO][PID:2785715] Checking for cancel request
[2026-04-15 22:47:38,014][  INFO][PID:2785715] Checking for cancel request
[2026-04-15 22:47:43,017][  INFO][PID:2785715] Checking for cancel request
[2026-04-15 22:47:48,022][  INFO][PID:2785715] Checking for cancel request
[2026-04-15 22:47:53,026][  INFO][PID:2785715] Checking for cancel request
[2026-04-15 22:47:58,029][  INFO][PID:2785715] Checking for cancel request
[2026-04-15 22:48:03,035][  INFO][PID:2785715] Checking for cancel request
[2026-04-15 22:48:08,046][  INFO][PID:2785715] Checking for cancel request
[2026-04-15 22:48:13,055][  INFO][PID:2785715] Checking for cancel request
[2026-04-15 22:48:18,062][  INFO][PID:2785715] Checking for cancel request
[2026-04-15 22:48:23,077][  INFO][PID:2785715] Checking for cancel request
[2026-04-15 22:48:28,091][  INFO][PID:2785715] Checking for cancel request
[2026-04-15 22:48:33,099][  INFO][PID:2785715] Checking for cancel request
[2026-04-15 22:48:38,102][  INFO][PID:2785715] Checking for cancel request
[2026-04-15 22:48:43,109][  INFO][PID:2785715] Checking for cancel request
[2026-04-15 22:48:48,118][  INFO][PID:2785715] Checking for cancel request
[2026-04-15 22:48:53,126][  INFO][PID:2785715] Checking for cancel request
[2026-04-15 22:48:58,132][  INFO][PID:2785715] Checking for cancel request
[2026-04-15 22:49:03,139][  INFO][PID:2785715] Checking for cancel request
[2026-04-15 22:49:08,146][  INFO][PID:2785715] Checking for cancel request
[2026-04-15 22:49:11,058][  INFO][PID:2785715] Allocated host ResallocHost, ticket_id=3259, hostname=2620:52:6:1161:dead:beef:cafe:c902, name=vmhost_p09_03_dev_02147499_20260415_224439, requested_tags=['arch_noarch', 'copr_builder']
[2026-04-15 22:49:11,063][  INFO][PID:2785715] Allocating ssh connection to builder
[2026-04-15 22:49:11,064][  INFO][PID:2785715] Checking that builder machine is OK
[2026-04-15 22:49:12,103][  INFO][PID:2785715] Installed copr-rpmbuild version: 1.6
[2026-04-15 22:49:12,105][ ERROR][PID:2785715] Re-try request for task on 'ResallocHost, ticket_id=3259, hostname=2620:52:6:1161:dead:beef:cafe:c902, name=vmhost_p09_03_dev_02147499_20260415_224439, requested_tags=['arch_noarch', 'copr_builder']': Minimum version for builder is 1.6.1
[2026-04-15 22:49:12,108][  INFO][PID:2785715] VM Release request
[2026-04-15 22:49:12,144][  INFO][PID:2785715] Retry #3 (on other host)
[2026-04-15 22:49:12,145][  INFO][PID:2785715] Checking for cancel request
[2026-04-15 22:49:12,147][  INFO][PID:2785715] VM allocation process starts
[2026-04-15 22:49:12,203][  INFO][PID:2785715] Trying to allocate VM: ResallocHost, ticket_id=3280, requested_tags=['arch_noarch', 'copr_builder']
[2026-04-15 22:49:15,284][  INFO][PID:2785715] Allocated host ResallocHost, ticket_id=3280, hostname=2620:52:6:1161:dead:beef:cafe:c902, name=vmhost_p09_03_dev_02147499_20260415_224439, requested_tags=['arch_noarch', 'copr_builder']
[2026-04-15 22:49:15,292][  INFO][PID:2785715] Allocating ssh connection to builder
[2026-04-15 22:49:15,292][  INFO][PID:2785715] Checking that builder machine is OK
[2026-04-15 22:49:15,475][  INFO][PID:2785715] Installed copr-rpmbuild version: 1.6
[2026-04-15 22:49:15,481][ ERROR][PID:2785715] Re-try request for task on 'ResallocHost, ticket_id=3280, hostname=2620:52:6:1161:dead:beef:cafe:c902, name=vmhost_p09_03_dev_02147499_20260415_224439, requested_tags=['arch_noarch', 'copr_builder']': Minimum version for builder is 1.6.1
[2026-04-15 22:49:15,488][  INFO][PID:2785715] VM Release request
[2026-04-15 22:49:15,555][  INFO][PID:2785715] Retry #4 (on other host)
[2026-04-15 22:49:15,556][  INFO][PID:2785715] Checking for cancel request
[2026-04-15 22:49:15,560][  INFO][PID:2785715] VM allocation process starts
[2026-04-15 22:49:15,677][  INFO][PID:2785715] Trying to allocate VM: ResallocHost, ticket_id=3281, requested_tags=['arch_noarch', 'copr_builder']
[2026-04-15 22:49:20,698][  INFO][PID:2785715] Checking for cancel request
[2026-04-15 22:49:25,715][  INFO][PID:2785715] Checking for cancel request
[2026-04-15 22:49:30,729][  INFO][PID:2785715] Checking for cancel request
[2026-04-15 22:49:35,742][  INFO][PID:2785715] Checking for cancel request
[2026-04-15 22:49:40,743][  INFO][PID:2785715] Checking for cancel request
[2026-04-15 22:49:45,745][  INFO][PID:2785715] Checking for cancel request
[2026-04-15 22:49:50,749][  INFO][PID:2785715] Checking for cancel request
[2026-04-15 22:49:55,754][  INFO][PID:2785715] Checking for cancel request
[2026-04-15 22:50:00,759][  INFO][PID:2785715] Checking for cancel request
[2026-04-15 22:50:05,764][  INFO][PID:2785715] Checking for cancel request
[2026-04-15 22:50:10,774][  INFO][PID:2785715] Checking for cancel request
[2026-04-15 22:50:15,776][  INFO][PID:2785715] Checking for cancel request
[2026-04-15 22:50:20,780][  INFO][PID:2785715] Checking for cancel request
[2026-04-15 22:50:25,788][  INFO][PID:2785715] Checking for cancel request
[2026-04-15 22:50:30,789][  INFO][PID:2785715] Checking for cancel request
[2026-04-15 22:50:35,796][  INFO][PID:2785715] Checking for cancel request
[2026-04-15 22:50:40,801][  INFO][PID:2785715] Checking for cancel request
[2026-04-15 22:50:45,803][  INFO][PID:2785715] Checking for cancel request
[2026-04-15 22:50:50,810][  INFO][PID:2785715] Checking for cancel request
[2026-04-15 22:50:55,815][  INFO][PID:2785715] Checking for cancel request
[2026-04-15 22:51:00,818][  INFO][PID:2785715] Checking for cancel request
[2026-04-15 22:51:04,031][  INFO][PID:2785715] Allocated host ResallocHost, ticket_id=3281, hostname=2620:52:6:1161:dead:beef:cafe:c900, name=vmhost_p09_01_dev_02147513_20260415_224723, requested_tags=['arch_noarch', 'copr_builder']
[2026-04-15 22:51:04,032][  INFO][PID:2785715] Allocating ssh connection to builder
[2026-04-15 22:51:04,035][  INFO][PID:2785715] Checking that builder machine is OK
[2026-04-15 22:51:05,000][  INFO][PID:2785715] Installed copr-rpmbuild version: 1.6
[2026-04-15 22:51:05,001][ ERROR][PID:2785715] Re-try request for task on 'ResallocHost, ticket_id=3281, hostname=2620:52:6:1161:dead:beef:cafe:c900, name=vmhost_p09_01_dev_02147513_20260415_224723, requested_tags=['arch_noarch', 'copr_builder']': Minimum version for builder is 1.6.1
[2026-04-15 22:51:05,002][  INFO][PID:2785715] VM Release request
[2026-04-15 22:51:05,012][ ERROR][PID:2785715] Backend process error: Three host tried without success: {'2620:52:6:1161:dead:beef:cafe:c902', '2620:52:6:1161:dead:beef:cafe:c903', '2620:52:6:1161:dead:beef:cafe:c900'}
[2026-04-15 22:51:05,014][WARNING][PID:2785715] Switching not-finished job state to 'failed'
[2026-04-15 22:51:05,017][  INFO][PID:2785715] Worker failed build, took 578.4404399394989
[2026-04-15 22:51:05,018][  INFO][PID:2785715] Sending build state back to frontend: {
    "builds": [
        {
            "timeout": 108000,
            "frontend_base_url": "https://copr.stg.fedoraproject.org",
            "memory_reqs": null,
            "enable_net": true,
            "project_owner": "@copr",
            "project_name": "TEST1776291979782039670-Architectures",
            "project_dirname": "TEST1776291979782039670-Architectures",
            "submitter": "frostyx",
            "ended_on": 1776293465.0147078,
            "started_on": 1776292886.5742679,
            "submitted_on": null,
            "status": 0,
            "chroot": "srpm-builds",
            "arch": null,
            "buildroot_pkgs": null,
            "task_id": "2925224",
            "build_id": 2925224,
            "package_name": null,
            "package_version": null,
            "git_repo": null,
            "git_hash": null,
            "git_branch": null,
            "source_type": 1,
            "source_json": "{\"url\": \"https://github.com/fedora-copr/copr-test-sources/raw/main/hello-2.8-1.src.rpm\"}",
            "pkg_name": null,
            "pkg_main_version": null,
            "pkg_epoch": null,
            "pkg_release": null,
            "srpm_url": null,
            "uses_devel_repo": null,
            "sandbox": "@copr/TEST1776291979782039670-Architectures--frostyx",
            "results": null,
            "appstream": false,
            "allow_user_ssh": null,
            "ssh_public_keys": null,
            "storage": null,
            "repos": [],
            "background": false,
            "distributions_in_build": [],
            "distributions_in_project": [
                "epel-9",
                "fedora-43"
            ],
            "destdir": "/var/lib/copr/public_html/results/@copr/TEST1776291979782039670-Architectures",
            "results_repo_url": "https://download.copr-dev.fedorainfracloud.org/results/@copr/TEST1776291979782039670-Architectures",
            "result_dir": "02925224",
            "built_packages": "",
            "tags": [
                "arch_noarch"
            ],
            "id": 2925224,
            "mockchain_macros": {
                "copr_username": "@copr",
                "copr_projectname": "TEST1776291979782039670-Architectures",
                "vendor": "Fedora Project COPR (@copr/TEST1776291979782039670-Architectures)"
            }
        }
    ]
}
[2026-04-15 22:51:05,090][  INFO][PID:2785715] Sending fedora-messaging bus message in build.end
[2026-04-15 22:51:06,937][  INFO][PID:2785715] Compressing /var/lib/copr/public_html/results/@copr/TEST1776291979782039670-Architectures/srpm-builds/02925224/builder-live.log by gzip
[2026-04-15 22:51:06,945][  INFO][PID:2785715] Running command 'gzip /var/lib/copr/public_html/results/@copr/TEST1776291979782039670-Architectures/srpm-builds/02925224/builder-live.log' as PID 2807724
[2026-04-15 22:51:06,955][  INFO][PID:2785715] Finished after 0 seconds with exit code 1 (gzip /var/lib/copr/public_html/results/@copr/TEST1776291979782039670-Architectures/srpm-builds/02925224/builder-live.log)
stdout:

stderr:
gzip: /var/lib/copr/public_html/results/@copr/TEST1776291979782039670-Architectures/srpm-builds/02925224/builder-live.log: No such file or directory


[2026-04-15 22:51:06,957][ ERROR][PID:2785715] Unable to compress file /var/lib/copr/public_html/results/@copr/TEST1776291979782039670-Architectures/srpm-builds/02925224/builder-live.log
[2026-04-15 22:51:06,959][  INFO][PID:2785715] Compressing /var/lib/copr/public_html/results/@copr/TEST1776291979782039670-Architectures/srpm-builds/02925224/backend.log by gzip
[2026-04-15 22:51:06,967][  INFO][PID:2785715] Running command 'gzip /var/lib/copr/public_html/results/@copr/TEST1776291979782039670-Architectures/srpm-builds/02925224/backend.log' as PID 2807726