[2026-04-15 22:27:22,910][ INFO][PID:2755737] Marking build as starting [2026-04-15 22:27:23,033][ INFO][PID:2755737] Checking for cancel request [2026-04-15 22:27:23,040][ INFO][PID:2755737] VM allocation process starts [2026-04-15 22:27:23,088][ INFO][PID:2755737] Trying to allocate VM: ResallocHost, ticket_id=2857, requested_tags=['arch_noarch', 'copr_builder'] [2026-04-15 22:27:28,094][ INFO][PID:2755737] Checking for cancel request [2026-04-15 22:27:33,099][ INFO][PID:2755737] Checking for cancel request [2026-04-15 22:27:38,102][ INFO][PID:2755737] Checking for cancel request [2026-04-15 22:27:43,107][ INFO][PID:2755737] Checking for cancel request [2026-04-15 22:27:48,110][ INFO][PID:2755737] Checking for cancel request [2026-04-15 22:27:53,112][ INFO][PID:2755737] Checking for cancel request [2026-04-15 22:27:58,116][ INFO][PID:2755737] Checking for cancel request [2026-04-15 22:28:03,119][ INFO][PID:2755737] Checking for cancel request [2026-04-15 22:28:08,131][ INFO][PID:2755737] Checking for cancel request [2026-04-15 22:28:13,135][ INFO][PID:2755737] Checking for cancel request [2026-04-15 22:28:18,143][ INFO][PID:2755737] Checking for cancel request [2026-04-15 22:28:23,145][ INFO][PID:2755737] Checking for cancel request [2026-04-15 22:28:28,149][ INFO][PID:2755737] Checking for cancel request [2026-04-15 22:28:33,155][ INFO][PID:2755737] Checking for cancel request [2026-04-15 22:28:38,160][ INFO][PID:2755737] Checking for cancel request [2026-04-15 22:28:43,163][ INFO][PID:2755737] Checking for cancel request [2026-04-15 22:28:48,169][ INFO][PID:2755737] Checking for cancel request [2026-04-15 22:28:53,183][ INFO][PID:2755737] Checking for cancel request [2026-04-15 22:28:58,185][ INFO][PID:2755737] Checking for cancel request [2026-04-15 22:29:03,189][ INFO][PID:2755737] Checking for cancel request [2026-04-15 22:29:08,194][ INFO][PID:2755737] Checking for cancel request [2026-04-15 22:29:13,198][ INFO][PID:2755737] Checking for cancel request [2026-04-15 22:29:18,207][ INFO][PID:2755737] Checking for cancel request [2026-04-15 22:29:23,218][ INFO][PID:2755737] Checking for cancel request [2026-04-15 22:29:28,226][ INFO][PID:2755737] Checking for cancel request [2026-04-15 22:29:33,233][ INFO][PID:2755737] Checking for cancel request [2026-04-15 22:29:38,235][ INFO][PID:2755737] Checking for cancel request [2026-04-15 22:29:43,236][ INFO][PID:2755737] Checking for cancel request [2026-04-15 22:29:48,238][ INFO][PID:2755737] Checking for cancel request [2026-04-15 22:29:53,241][ INFO][PID:2755737] Checking for cancel request [2026-04-15 22:29:58,242][ INFO][PID:2755737] Checking for cancel request [2026-04-15 22:30:03,250][ INFO][PID:2755737] Checking for cancel request [2026-04-15 22:30:08,258][ INFO][PID:2755737] Checking for cancel request [2026-04-15 22:30:13,263][ INFO][PID:2755737] Checking for cancel request [2026-04-15 22:30:18,265][ INFO][PID:2755737] Checking for cancel request [2026-04-15 22:30:23,268][ INFO][PID:2755737] Checking for cancel request [2026-04-15 22:30:28,271][ INFO][PID:2755737] Checking for cancel request [2026-04-15 22:30:33,272][ INFO][PID:2755737] Checking for cancel request [2026-04-15 22:30:38,275][ INFO][PID:2755737] Checking for cancel request [2026-04-15 22:30:43,278][ INFO][PID:2755737] Checking for cancel request [2026-04-15 22:30:48,286][ INFO][PID:2755737] Checking for cancel request [2026-04-15 22:30:53,289][ INFO][PID:2755737] Checking for cancel request [2026-04-15 22:30:58,293][ INFO][PID:2755737] Checking for cancel request [2026-04-15 22:31:03,296][ INFO][PID:2755737] Checking for cancel request [2026-04-15 22:31:08,300][ INFO][PID:2755737] Checking for cancel request [2026-04-15 22:31:13,304][ INFO][PID:2755737] Checking for cancel request [2026-04-15 22:31:18,306][ INFO][PID:2755737] Checking for cancel request [2026-04-15 22:31:23,308][ INFO][PID:2755737] Checking for cancel request [2026-04-15 22:31:28,311][ INFO][PID:2755737] Checking for cancel request [2026-04-15 22:31:33,316][ INFO][PID:2755737] Checking for cancel request [2026-04-15 22:31:38,319][ INFO][PID:2755737] Checking for cancel request [2026-04-15 22:31:41,402][ INFO][PID:2755737] Allocated host ResallocHost, ticket_id=2857, hostname=54.89.222.203, name=aws_aarch64_spot_dev_02147444_20260415_222901, requested_tags=['arch_noarch', 'copr_builder'] [2026-04-15 22:31:41,402][ INFO][PID:2755737] Allocating ssh connection to builder [2026-04-15 22:31:41,403][ INFO][PID:2755737] Checking that builder machine is OK [2026-04-15 22:31:41,783][ INFO][PID:2755737] Installed copr-rpmbuild version: 1.6 [2026-04-15 22:31:41,784][ ERROR][PID:2755737] Re-try request for task on 'ResallocHost, ticket_id=2857, hostname=54.89.222.203, name=aws_aarch64_spot_dev_02147444_20260415_222901, requested_tags=['arch_noarch', 'copr_builder']': Minimum version for builder is 1.6.1 [2026-04-15 22:31:41,785][ INFO][PID:2755737] VM Release request [2026-04-15 22:31:41,796][ INFO][PID:2755737] Retry #1 (on other host) [2026-04-15 22:31:41,797][ INFO][PID:2755737] Checking for cancel request [2026-04-15 22:31:41,798][ INFO][PID:2755737] VM allocation process starts [2026-04-15 22:31:41,834][ INFO][PID:2755737] Trying to allocate VM: ResallocHost, ticket_id=2936, requested_tags=['arch_noarch', 'copr_builder'] [2026-04-15 22:31:46,838][ INFO][PID:2755737] Checking for cancel request [2026-04-15 22:31:51,841][ INFO][PID:2755737] Checking for cancel request [2026-04-15 22:31:56,844][ INFO][PID:2755737] Checking for cancel request [2026-04-15 22:32:01,846][ INFO][PID:2755737] Checking for cancel request [2026-04-15 22:32:06,848][ INFO][PID:2755737] Checking for cancel request [2026-04-15 22:32:11,851][ INFO][PID:2755737] Checking for cancel request [2026-04-15 22:32:16,857][ INFO][PID:2755737] Checking for cancel request [2026-04-15 22:32:21,863][ INFO][PID:2755737] Checking for cancel request [2026-04-15 22:32:26,867][ INFO][PID:2755737] Checking for cancel request [2026-04-15 22:32:31,869][ INFO][PID:2755737] Checking for cancel request [2026-04-15 22:32:36,873][ INFO][PID:2755737] Checking for cancel request [2026-04-15 22:32:41,878][ INFO][PID:2755737] Checking for cancel request [2026-04-15 22:32:46,880][ INFO][PID:2755737] Checking for cancel request [2026-04-15 22:32:51,883][ INFO][PID:2755737] Checking for cancel request [2026-04-15 22:32:56,886][ INFO][PID:2755737] Checking for cancel request [2026-04-15 22:33:01,888][ INFO][PID:2755737] Checking for cancel request [2026-04-15 22:33:06,891][ INFO][PID:2755737] Checking for cancel request [2026-04-15 22:33:11,900][ INFO][PID:2755737] Checking for cancel request [2026-04-15 22:33:16,903][ INFO][PID:2755737] Checking for cancel request [2026-04-15 22:33:21,906][ INFO][PID:2755737] Checking for cancel request [2026-04-15 22:33:26,925][ INFO][PID:2755737] Checking for cancel request [2026-04-15 22:33:31,928][ INFO][PID:2755737] Checking for cancel request [2026-04-15 22:33:36,937][ INFO][PID:2755737] Checking for cancel request [2026-04-15 22:33:41,939][ INFO][PID:2755737] Checking for cancel request [2026-04-15 22:33:46,941][ INFO][PID:2755737] Checking for cancel request [2026-04-15 22:33:51,953][ INFO][PID:2755737] Checking for cancel request [2026-04-15 22:33:56,970][ INFO][PID:2755737] Checking for cancel request [2026-04-15 22:34:01,983][ INFO][PID:2755737] Checking for cancel request [2026-04-15 22:34:06,985][ INFO][PID:2755737] Checking for cancel request [2026-04-15 22:34:11,989][ INFO][PID:2755737] Checking for cancel request [2026-04-15 22:34:16,991][ INFO][PID:2755737] Checking for cancel request [2026-04-15 22:34:22,002][ INFO][PID:2755737] Checking for cancel request [2026-04-15 22:34:27,004][ INFO][PID:2755737] Checking for cancel request [2026-04-15 22:34:32,009][ INFO][PID:2755737] Checking for cancel request [2026-04-15 22:34:37,022][ INFO][PID:2755737] Checking for cancel request [2026-04-15 22:34:42,032][ INFO][PID:2755737] Checking for cancel request [2026-04-15 22:34:47,039][ INFO][PID:2755737] Checking for cancel request [2026-04-15 22:34:52,053][ INFO][PID:2755737] Checking for cancel request [2026-04-15 22:34:57,068][ INFO][PID:2755737] Checking for cancel request [2026-04-15 22:35:00,328][ INFO][PID:2755737] Allocated host ResallocHost, ticket_id=2936, hostname=2620:52:6:1161:dead:beef:cafe:c904, name=vmhost_x86_04_dev_02147454_20260415_223159, requested_tags=['arch_noarch', 'copr_builder'] [2026-04-15 22:35:00,340][ INFO][PID:2755737] Allocating ssh connection to builder [2026-04-15 22:35:00,352][ INFO][PID:2755737] Checking that builder machine is OK [2026-04-15 22:35:01,188][ INFO][PID:2755737] Installed copr-rpmbuild version: 1.6 [2026-04-15 22:35:01,225][ ERROR][PID:2755737] Re-try request for task on 'ResallocHost, ticket_id=2936, hostname=2620:52:6:1161:dead:beef:cafe:c904, name=vmhost_x86_04_dev_02147454_20260415_223159, requested_tags=['arch_noarch', 'copr_builder']': Minimum version for builder is 1.6.1 [2026-04-15 22:35:01,229][ INFO][PID:2755737] VM Release request [2026-04-15 22:35:01,292][ INFO][PID:2755737] Retry #2 (on other host) [2026-04-15 22:35:01,299][ INFO][PID:2755737] Checking for cancel request [2026-04-15 22:35:01,306][ INFO][PID:2755737] VM allocation process starts [2026-04-15 22:35:01,382][ INFO][PID:2755737] Trying to allocate VM: ResallocHost, ticket_id=3038, requested_tags=['arch_noarch', 'copr_builder'] [2026-04-15 22:35:06,396][ INFO][PID:2755737] Checking for cancel request [2026-04-15 22:35:11,403][ INFO][PID:2755737] Checking for cancel request [2026-04-15 22:35:16,409][ INFO][PID:2755737] Checking for cancel request [2026-04-15 22:35:21,411][ INFO][PID:2755737] Checking for cancel request [2026-04-15 22:35:26,418][ INFO][PID:2755737] Checking for cancel request [2026-04-15 22:35:31,424][ INFO][PID:2755737] Checking for cancel request [2026-04-15 22:35:36,431][ INFO][PID:2755737] Checking for cancel request [2026-04-15 22:35:41,440][ INFO][PID:2755737] Checking for cancel request [2026-04-15 22:35:46,452][ INFO][PID:2755737] Checking for cancel request [2026-04-15 22:35:51,460][ INFO][PID:2755737] Checking for cancel request [2026-04-15 22:35:56,465][ INFO][PID:2755737] Checking for cancel request [2026-04-15 22:36:01,480][ INFO][PID:2755737] Checking for cancel request [2026-04-15 22:36:06,491][ INFO][PID:2755737] Checking for cancel request [2026-04-15 22:36:11,509][ INFO][PID:2755737] Checking for cancel request [2026-04-15 22:36:16,518][ INFO][PID:2755737] Checking for cancel request [2026-04-15 22:36:21,527][ INFO][PID:2755737] Checking for cancel request [2026-04-15 22:36:26,537][ INFO][PID:2755737] Checking for cancel request [2026-04-15 22:36:31,540][ INFO][PID:2755737] Checking for cancel request [2026-04-15 22:36:36,550][ INFO][PID:2755737] Checking for cancel request [2026-04-15 22:36:41,554][ INFO][PID:2755737] Checking for cancel request [2026-04-15 22:36:46,557][ INFO][PID:2755737] Checking for cancel request [2026-04-15 22:36:51,564][ INFO][PID:2755737] Checking for cancel request [2026-04-15 22:36:56,568][ INFO][PID:2755737] Checking for cancel request [2026-04-15 22:37:01,576][ INFO][PID:2755737] Checking for cancel request [2026-04-15 22:37:06,587][ INFO][PID:2755737] Checking for cancel request [2026-04-15 22:37:11,591][ INFO][PID:2755737] Checking for cancel request [2026-04-15 22:37:16,601][ INFO][PID:2755737] Checking for cancel request [2026-04-15 22:37:21,608][ INFO][PID:2755737] Checking for cancel request [2026-04-15 22:37:26,612][ INFO][PID:2755737] Checking for cancel request [2026-04-15 22:37:31,622][ INFO][PID:2755737] Checking for cancel request [2026-04-15 22:37:36,624][ INFO][PID:2755737] Checking for cancel request [2026-04-15 22:37:41,628][ INFO][PID:2755737] Checking for cancel request [2026-04-15 22:37:46,630][ INFO][PID:2755737] Checking for cancel request [2026-04-15 22:37:51,632][ INFO][PID:2755737] Checking for cancel request [2026-04-15 22:37:56,634][ INFO][PID:2755737] Checking for cancel request [2026-04-15 22:38:01,636][ INFO][PID:2755737] Checking for cancel request [2026-04-15 22:38:06,639][ INFO][PID:2755737] Checking for cancel request [2026-04-15 22:38:11,642][ INFO][PID:2755737] Checking for cancel request [2026-04-15 22:38:16,645][ INFO][PID:2755737] Checking for cancel request [2026-04-15 22:38:21,648][ INFO][PID:2755737] Checking for cancel request [2026-04-15 22:38:26,653][ INFO][PID:2755737] Checking for cancel request [2026-04-15 22:38:31,655][ INFO][PID:2755737] Checking for cancel request [2026-04-15 22:38:36,659][ INFO][PID:2755737] Checking for cancel request [2026-04-15 22:38:41,662][ INFO][PID:2755737] Checking for cancel request [2026-04-15 22:38:46,666][ INFO][PID:2755737] Checking for cancel request [2026-04-15 22:38:51,671][ INFO][PID:2755737] Checking for cancel request [2026-04-15 22:38:56,674][ INFO][PID:2755737] Checking for cancel request [2026-04-15 22:39:01,678][ INFO][PID:2755737] Checking for cancel request [2026-04-15 22:39:06,681][ INFO][PID:2755737] Checking for cancel request [2026-04-15 22:39:11,684][ INFO][PID:2755737] Checking for cancel request [2026-04-15 22:39:16,688][ INFO][PID:2755737] Checking for cancel request [2026-04-15 22:39:19,971][ INFO][PID:2755737] Allocated host ResallocHost, ticket_id=3038, hostname=18.207.122.79, name=aws_aarch64_spot_dev_02147471_20260415_223728, requested_tags=['arch_noarch', 'copr_builder'] [2026-04-15 22:39:19,974][ INFO][PID:2755737] Allocating ssh connection to builder [2026-04-15 22:39:19,977][ INFO][PID:2755737] Checking that builder machine is OK [2026-04-15 22:39:20,457][ INFO][PID:2755737] Installed copr-rpmbuild version: 1.6 [2026-04-15 22:39:20,458][ ERROR][PID:2755737] Re-try request for task on 'ResallocHost, ticket_id=3038, hostname=18.207.122.79, name=aws_aarch64_spot_dev_02147471_20260415_223728, requested_tags=['arch_noarch', 'copr_builder']': Minimum version for builder is 1.6.1 [2026-04-15 22:39:20,465][ INFO][PID:2755737] VM Release request [2026-04-15 22:39:20,510][ ERROR][PID:2755737] Backend process error: Three host tried without success: {'54.89.222.203', '18.207.122.79', '2620:52:6:1161:dead:beef:cafe:c904'} [2026-04-15 22:39:20,521][WARNING][PID:2755737] Switching not-finished job state to 'failed' [2026-04-15 22:39:20,524][ INFO][PID:2755737] Worker failed build, took 717.6118831634521 [2026-04-15 22:39:20,543][ INFO][PID:2755737] Sending build state back to frontend: { "builds": [ { "timeout": 108000, "frontend_base_url": "https://copr.stg.fedoraproject.org", "memory_reqs": null, "enable_net": true, "project_owner": "@copr", "project_name": "TEST1776291979780950227Appstream", "project_dirname": "TEST1776291979780950227Appstream", "submitter": "frostyx", "ended_on": 1776292760.5146222, "started_on": 1776292042.902739, "submitted_on": null, "status": 0, "chroot": "srpm-builds", "arch": null, "buildroot_pkgs": null, "task_id": "2925214", "build_id": 2925214, "package_name": null, "package_version": null, "git_repo": null, "git_hash": null, "git_branch": null, "source_type": 1, "source_json": "{\"url\": \"https://github.com/fedora-copr/copr-test-sources/raw/main/hello-2.8-1.src.rpm\"}", "pkg_name": null, "pkg_main_version": null, "pkg_epoch": null, "pkg_release": null, "srpm_url": null, "uses_devel_repo": null, "sandbox": "@copr/TEST1776291979780950227Appstream--frostyx", "results": null, "appstream": false, "allow_user_ssh": null, "ssh_public_keys": null, "storage": null, "repos": [], "background": false, "distributions_in_build": [], "distributions_in_project": [ "fedora-43" ], "destdir": "/var/lib/copr/public_html/results/@copr/TEST1776291979780950227Appstream", "results_repo_url": "https://download.copr-dev.fedorainfracloud.org/results/@copr/TEST1776291979780950227Appstream", "result_dir": "02925214", "built_packages": "", "tags": [ "arch_noarch" ], "id": 2925214, "mockchain_macros": { "copr_username": "@copr", "copr_projectname": "TEST1776291979780950227Appstream", "vendor": "Fedora Project COPR (@copr/TEST1776291979780950227Appstream)" } } ] } [2026-04-15 22:39:21,289][ INFO][PID:2755737] Sending fedora-messaging bus message in build.end [2026-04-15 22:39:27,258][ INFO][PID:2755737] Compressing /var/lib/copr/public_html/results/@copr/TEST1776291979780950227Appstream/srpm-builds/02925214/builder-live.log by gzip [2026-04-15 22:39:27,266][ INFO][PID:2755737] Running command 'gzip /var/lib/copr/public_html/results/@copr/TEST1776291979780950227Appstream/srpm-builds/02925214/builder-live.log' as PID 2781047 [2026-04-15 22:39:27,269][ INFO][PID:2755737] Finished after 0 seconds with exit code 1 (gzip /var/lib/copr/public_html/results/@copr/TEST1776291979780950227Appstream/srpm-builds/02925214/builder-live.log) stdout: stderr: gzip: /var/lib/copr/public_html/results/@copr/TEST1776291979780950227Appstream/srpm-builds/02925214/builder-live.log: No such file or directory [2026-04-15 22:39:27,270][ ERROR][PID:2755737] Unable to compress file /var/lib/copr/public_html/results/@copr/TEST1776291979780950227Appstream/srpm-builds/02925214/builder-live.log [2026-04-15 22:39:27,272][ INFO][PID:2755737] Compressing /var/lib/copr/public_html/results/@copr/TEST1776291979780950227Appstream/srpm-builds/02925214/backend.log by gzip [2026-04-15 22:39:27,281][ INFO][PID:2755737] Running command 'gzip /var/lib/copr/public_html/results/@copr/TEST1776291979780950227Appstream/srpm-builds/02925214/backend.log' as PID 2781048